From b391a2322cf280b233278308b560d57813d67850 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Sun, 15 Nov 2026 17:14:23 +0100 Subject: [PATCH] =?UTF-8?q?hub=20(unreleased):=20review=20fix=202=20?= =?UTF-8?q?=E2=80=94=20the=20notice's=20daily=20limit=20counts=20failed=20?= =?UTF-8?q?tries=20too,=20and=20the=20wait=20between=20tries=20doubles=20(?= =?UTF-8?q?a=20send=20that=20errors=20after=20it=20was=20accepted)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS --- REUSE.md | 2 +- .../A/redproof-review-2.txt | 36 +++++++++++++ hub/CHANGELOG.md | 11 ++++ hub/internal/api/end_notice.go | 27 +++++++--- hub/internal/api/end_notice_review_test.go | 52 +++++++++++++++++++ hub/internal/api/end_notice_test.go | 4 +- hub/internal/notify/end_notice.go | 5 +- hub/internal/store/offsite_end_notice.go | 9 ++-- hub/internal/store/offsite_ended.go | 18 +++++-- 9 files changed, 146 insertions(+), 18 deletions(-) create mode 100644 documentation/audits/build-release-4b-2026-11-15/A/redproof-review-2.txt diff --git a/REUSE.md b/REUSE.md index 2da5265c..f2f74821 100644 --- a/REUSE.md +++ b/REUSE.md @@ -80,7 +80,7 @@ | `store.NoteAppList` / `AppPresences` + `offsitekeys.agreeEnded` (D72/D84, `07` §8.2f) | hub/internal/store/offsite_ended.go, hub/internal/offsitekeys/ended.go | `NoteAppList(customerID, deployed, claimed, now)`; `agreeEnded(now, presence, notices, listing, haveListing, claims)` | The hub's OWN record of when an app left a box's app list (fed by `handleReport`), and the decision which snapshot ids of ended apps a window may remove outside its cap | **The box's dates decide nothing.** An id is agreed only when its app has been absent 30 days by the hub's clock AND the hub's own listing shows the file older than that absence. `WindowCredit.CapExplains` is the cap's part, `EndedIDs` the agreed ids — a reader that explains ids by name must not also spend `Explains`. **After the review of 2026-11-15:** one window agrees to at most the cap's size from the hub's OWN count (`endedCap`/`boundEndedToCap`); a request for more than half of the store waits a week — its first sight is the hub's own row (`store.NoteLargeEndedRequest`, key `largeSetKey` = a hash of the whole app set), never a search of text a box could write; `store.NoteAppListClaims` returns never-seen apps once for the operator's warning; a claimed name that is not an app name is written nowhere (`ledgerName`). | | `notify.householdReceiptEvents` + `Dispatcher.SendEndNotice` (D84; the notice lock, `09` §3 decision 315) | hub/internal/notify/end_notice.go | register; `SendEndNotice(customerID, app, ends) (EndNoticeOutcome, why)` | The mail that tells the household about its own press (one type: `offsite_end_scheduled`). **The box's event mails nobody**: the hub sends the notice from its own state (`api.endNotices`, at every report, until the mail service accepts it) | Bypasses the household's group switches; the hub writes it from a checked app name and its OWN day. `SendEndNotice` keeps no state and tries once: only `EndNoticeSent` tells the household; the daily limit (5 per household, counted in the sent log) answers `EndNoticeLimit` — the caller tries again, it never drops. **Every new entry needs its own ruling** — pinned by `TestD84_Receipt_OpensNothingElse`, `TestNotice_*`. | | `store.NoteEndNotices` / `EndNotices` / `MarkEndNoticeTold` + `api.endNotices` (the notice lock, `07` §8.2f) | hub/internal/store/offsite_end_notice.go, hub/internal/api/end_notice.go | `NoteEndNotices(customerID, deployed, claims, tierOn, now)`; `EndNotices(customerID)`; `MarkEndNoticeTold(customerID, app, press, to, now)` | The hub's OWN record that a household (or the operator in its place) was told that an app's off-site copies will be deleted. `offsitekeys.agreeEnded` agrees only when it says „told” 30 days ago | **Written only by the hub, only after the mail service accepted the mail; never derived from the sent log or from a box's date.** `press` (the box's `removed_at`) is a key only: another value VOIDS the notice. An install, a take-back (a report that shows the tier ON and no record) and a household delete remove the row. A new condition of a deletion goes HERE, not into `notification_log`. | -| `Dispatcher.SendOperatorNow` | hub/internal/notify/end_notice.go | `SendOperatorNow(customerID, eventType, severity, message, detailsJSON) error` | An operator mail whose DELIVERY is a condition of something else: it answers only when the mail service has (nil = accepted) | Operator-only types only; no cooldown and no background retry — the caller keeps its own „told once” record and tries again, **and bounds itself** from the sent log (`store.CountOperatorSentSince`, `OperatorSentAboutSince`: per household a day, per thing a week — `api.endNoticeToOperator`). For every other operator mail use `ProcessEvent`. | +| `Dispatcher.SendOperatorNow` | hub/internal/notify/end_notice.go | `SendOperatorNow(customerID, eventType, severity, message, detailsJSON) error` | An operator mail whose DELIVERY is a condition of something else: it answers only when the mail service has (nil = accepted) | Operator-only types only; no cooldown and no background retry — the caller keeps its own „told once” record and tries again, **and bounds itself** from the sent log (`store.CountOperatorTriesSince` — sent AND failed tries — and `OperatorSentAboutSince`: per household a day, per thing a week — `api.endNoticeToOperator`). For every other operator mail use `ProcessEvent`. | | `accesslog.New` / `Middleware` / `Mark` (v0.158.0, R-1140) | hub/internal/accesslog/accesslog.go | `Mark(ctx, kind)` | The hub's own access log (`/access-log/`, 90 days) and marking the identity KIND an auth check established | Never log a value: the path only (no query; `/bind/` and 32+-char segments redacted). A new auth path calls `Mark` with its kind. A logging failure never fails a request. | | the timed SQLite driver `sqlite-timed` + `SetSlowStatementThreshold` (v0.178.0, R-1346) | hub/internal/store/timeddriver.go | `store.New` opens it — nothing to call | Names a statement over 1 s: `[WARN] slow statement: (over 1s): ` | The SQL text only, never a bound value. Put a heavy statement's SQL in a const and pin its plan (`r1346_speed.go`, `TestR1346_HeavyStatementsUseTheirIndex`): on the hub's network disk a whole-table read is seconds to minutes. | | `(*Store).SaveOneTimeSecret` / `OffsitePassword` / `SealLegacyOffsiteSecrets` | hub/internal/store/offsite_seal.go | — | Storing / reading the sub-account password | **Sealed AES-256-GCM; no key → refused (fail-closed).** Under `go test` every store gets a fixed key (`testing.Testing()`); production needs `OFFSITE_SECRET_KEY`. Never serve the value to a box. | diff --git a/documentation/audits/build-release-4b-2026-11-15/A/redproof-review-2.txt b/documentation/audits/build-release-4b-2026-11-15/A/redproof-review-2.txt new file mode 100644 index 00000000..b1984fb7 --- /dev/null +++ b/documentation/audits/build-release-4b-2026-11-15/A/redproof-review-2.txt @@ -0,0 +1,36 @@ +# Review finding 2 — a send that errors after it was accepted is mailed every hour, outside the daily limit. The test on the code BEFORE the fix: +--- FAIL: TestReview2_ASendThatErrorsAfterItWasAcceptedIsNotMailedEveryHour (0.11s) + --- FAIL: TestReview2_ASendThatErrorsAfterItWasAcceptedIsNotMailedEveryHour/the_household's_notice (0.06s) + end_notice_review_test.go:123: 12 hours of a send that errors after it was accepted: 12 mails, want at most 4 (tries at 0, 1, 3 and 7 hours) + --- FAIL: TestReview2_ASendThatErrorsAfterItWasAcceptedIsNotMailedEveryHour/the_operator's_in-place_notice (0.06s) + end_notice_review_test.go:123: 12 hours of a send that errors after it was accepted: 12 mails, want at most 4 (tries at 0, 1, 3 and 7 hours) +FAIL +FAIL gitea.dooplex.hu/admin/felhom-hub/internal/api 0.134s +FAIL + +# After the fix, by mutation: +== MUTATION: the wait does not grow with the tries + RED on the mutant, GREEN on the real file + --- FAIL: TestReview2_ASendThatErrorsAfterItWasAcceptedIsNotMailedEveryHour (0.10s) + end_notice_review_test.go:123: 12 hours of a send that errors after it was accepted: 5 mails, want at most 4 (tries at 0, 1, 3 and 7 hours) + end_notice_review_test.go:123: 12 hours of a send that errors after it was accepted: 5 mails, want at most 4 (tries at 0, 1, 3 and 7 hours) +== MUTATION: the operator's daily limit counts only sent mails + RED on the mutant, GREEN on the real file + --- FAIL: TestReview2_ASendThatErrorsAfterItWasAcceptedIsNotMailedEveryHour (0.12s) + end_notice_review_test.go:129: four days of it: 8 mails — the daily limit must count a try that failed (all of them are inside this test's real 24 hours) + --- FAIL: TestNotice_NoAddress_ABoxCannotFloodTheOperator (0.29s) + end_notice_test.go:543: 101 presses in a day mailed the operator 51 times, want the household's daily limit 5 + +2 mutations, 0 not a red-proof +== MUTATION: the household's daily limit counts only sent mails + RED on the mutant, GREEN on the real file + --- FAIL: TestReview2_ASendThatErrorsAfterItWasAcceptedIsNotMailedEveryHour (0.12s) + end_notice_review_test.go:129: four days of it: 8 mails — the daily limit must count a try that failed (all of them are inside this test's real 24 hours) + +1 mutations, 0 not a red-proof +== MUTATION: the operator's daily limit counts only sent mails (the query) + RED on the mutant, GREEN on the real file + --- FAIL: TestReview2_ASendThatErrorsAfterItWasAcceptedIsNotMailedEveryHour (0.12s) + end_notice_review_test.go:129: four days of it: 8 mails — the daily limit must count a try that failed (all of them are inside this test's real 24 hours) + +1 mutations, 0 not a red-proof diff --git a/hub/CHANGELOG.md b/hub/CHANGELOG.md index 1749ce59..a270791f 100644 --- a/hub/CHANGELOG.md +++ b/hub/CHANGELOG.md @@ -1,5 +1,16 @@ ## Unreleased (after v0.182.0) +- **Review of the notice lock, finding 2 (2026-11-15 night): a send that errors AFTER the mail service accepted it is + not mailed every hour.** A client timeout is logged `failed` although the mail may have gone; the notice was tried + again every hour and the daily limit counted only `sent` rows — 12 household mails in 12 hours in the review's + test. Now (a) the daily limit counts every try that reached the mail service, `sent` and `failed` + (`store.CountCustomerTriesSince`, `CountOperatorTriesSince` — the latter replaces `CountOperatorSentSince`), for + the household's notice and for the operator's in-place mail; (b) the wait before the next try doubles with the + tries: 1 h, 2 h, 4 h, 8 h, 16 h, then 24 h (`api.endNoticeGap`). The „stuck” line after 3 days still comes. **Cost, + said plainly:** after a real outage of the mail service the notice goes up to a day later than before, and a + household that already had 5 failed tries in 24 hours waits for the limit. Test + `TestReview2_ASendThatErrorsAfterItWasAcceptedIsNotMailedEveryHour` (both paths: at most 4 mails in 12 hours, 5 in + four days, never „told”, one stuck line); seen red before the fix (12 mails), 4 mutations red. - **Review of the notice lock, finding 4 (2026-11-15 night): „told” is the time AFTER the send returned.** One walk used its start time for every row, so with many rows (or a slow mail service) the last row was „told” minutes before its mail was accepted, and the 30 days counted from before the mail. `api.endNotices` now reads its clock diff --git a/hub/internal/api/end_notice.go b/hub/internal/api/end_notice.go index 40b43ccc..a584b897 100644 --- a/hub/internal/api/end_notice.go +++ b/hub/internal/api/end_notice.go @@ -24,15 +24,18 @@ import ( // (`offsite_end_notice_operator`), and that mail starts the 30 days. When an address appears later, the household // gets its own mail and the 30 days start again from it. // - the daily limit, the mail hold, a send that fails → nothing is told; the reason is written on the row (the -// window's answer names it) and the next try is at a report at least endNoticeRetryGap later. +// window's answer names it) and the next try is at a report at least endNoticeGap(tries) later (1 h, 2 h, 4 h …). // - a row that has not been told for EndNoticeStuckAfter → one line to the operator (`offsite_end_notice_stuck`). // // Nothing here reads a date a box sent, and no event of a box starts or stops anything. const ( - // endNoticeRetryGap: after a try that told nobody, the next try waits this long (a report comes every 15 minutes; - // a mail service that is down is not asked 96 times a day per app). - endNoticeRetryGap = time.Hour + // endNoticeRetryGap: after the FIRST try that told nobody, the next try waits this long (a report comes every 15 + // minutes; a mail service that is down is not asked 96 times a day per app). It doubles with every further try + // and stops at endNoticeRetryGapMax (endNoticeGap): a send that errors AFTER the mail service accepted it is a + // mail each time, so the tries must thin out (review of 2026-11-15, finding 2). + endNoticeRetryGap = time.Hour + endNoticeRetryGapMax = 24 * time.Hour // EndNoticeStuckAfter: how long a notice may stay unsent before the operator gets one line. THREE DAYS: longer // than any outage of the mail service the hub has had and than the hold on a restored copy is meant to last, and @@ -42,6 +45,18 @@ const ( EndNoticeStuckAfter = 3 * 24 * time.Hour ) +// endNoticeGap is the wait after the tries-th try that told nobody: 1 h, 2 h, 4 h, 8 h, 16 h, then 24 h. +func endNoticeGap(tries int) time.Duration { + g := endNoticeRetryGap + for i := 1; i < tries && g < endNoticeRetryGapMax; i++ { + g *= 2 + } + if g > endNoticeRetryGapMax { + g = endNoticeRetryGapMax + } + return g +} + // endNoticeSpawn runs the sends off the report's request (a send can wait 10 s on the mail service). A seam: tests // run it in line. var endNoticeSpawn = func(f func()) { go f() } @@ -97,7 +112,7 @@ func (h *Handler) endNotices(customerID string, start time.Time) { continue } now := clock() - if n.LastReason != "" && !n.LastTry.IsZero() && now.Sub(n.LastTry) < endNoticeRetryGap { + if n.LastReason != "" && !n.LastTry.IsZero() && now.Sub(n.LastTry) < endNoticeGap(n.Tries) { h.endNoticeStuck(customerID, n, now) continue } @@ -162,7 +177,7 @@ func (h *Handler) endNoticeToOperator(customerID string, n store.EndNotice, why } else if again { return time.Time{}, fmt.Errorf("the operator was told about this app of this household less than %d days ago — the notice for this press goes later", int(operatorNoticeSameAppGap.Hours()/24)) } - if sent, err := h.store.CountOperatorSentSince(customerID, offsitekeys.EventEndNoticeOperator, wall.Add(-24*time.Hour)); err != nil { + if sent, err := h.store.CountOperatorTriesSince(customerID, offsitekeys.EventEndNoticeOperator, wall.Add(-24*time.Hour)); err != nil { return time.Time{}, fmt.Errorf("the hub's sent log could not be read") } else if sent >= operatorNoticesPerDay { return time.Time{}, fmt.Errorf("the operator already got %d of these notices about this household in the last 24 hours (the daily limit) — this one goes later", operatorNoticesPerDay) diff --git a/hub/internal/api/end_notice_review_test.go b/hub/internal/api/end_notice_review_test.go index 91e68873..0cf37970 100644 --- a/hub/internal/api/end_notice_review_test.go +++ b/hub/internal/api/end_notice_review_test.go @@ -8,6 +8,7 @@ import ( "testing" "time" + "gitea.dooplex.hu/admin/felhom-hub/internal/offsitekeys" "gitea.dooplex.hu/admin/felhom-hub/internal/store" ) @@ -85,3 +86,54 @@ func TestReview4_ToldIsTheTimeAfterTheSendReturned(t *testing.T) { n.ToldAt.Format(time.RFC3339), start.UTC().Format(time.RFC3339)) } } + +// Finding 2. A send that ERRORS AFTER the mail service accepted it (a client timeout) was mailed again every hour, +// and it was outside the daily limit, because the limit counted only `sent` rows: 12 mails in 12 hours. Now the +// limit counts every try that reached the mail service (`sent` + `failed`), and the wait before the next try grows +// with the tries (1 h, 2 h, 4 h … at most 24 h). The operator's in-place mail has the same shape and the same +// fix. The „stuck” line after 3 days still comes. +func TestReview2_ASendThatErrorsAfterItWasAcceptedIsNotMailedEveryHour(t *testing.T) { + for _, c := range []struct{ name, email, to string }{ + {"the household's notice", household, household}, + {"the operator's in-place notice", "", operator}, + } { + t.Run(c.name, func(t *testing.T) { + r := newNoticeRig(t, c.email) + r.d.SetSendFnForTest(func(to, subject, body string, _ map[string]string) error { + r.mu.Lock() + defer r.mu.Unlock() + r.mails = append(r.mails, noticeMail{to, subject, body}) // the mail WENT … + return errors.New("context deadline exceeded") // … and the client gave up waiting + }) + count := func() int { + n := 0 + for _, m := range r.to(c.to) { + if !strings.Contains(m.subject, "offsite_end_notice_stuck") && !strings.Contains(m.subject, "offsite_end_claim_unseen") { + n++ + } + } + return n + } + r.report(repPressed) + now := time.Now() + for i := 1; i <= 11; i++ { // a report every hour for twelve hours + r.h.endNotices("c1", now.Add(time.Duration(i)*61*time.Minute)) + } + if n := count(); n > 4 { + t.Fatalf("12 hours of a send that errors after it was accepted: %d mails, want at most 4 (tries at 0, 1, 3 and 7 hours)", n) + } + for i := 12; i <= 100; i++ { // … and four days of it + r.h.endNotices("c1", now.Add(time.Duration(i)*61*time.Minute)) + } + if n := count(); n > 5 { + t.Fatalf("four days of it: %d mails — the daily limit must count a try that failed (all of them are inside this test's real 24 hours)", n) + } + if n, _ := r.notice("vikunja"); n.Told() { + t.Fatalf("a send that returned an error counts as told: %+v", n) + } + if e := r.events(offsitekeys.EventEndNoticeStuck); len(e) != 1 { + t.Fatalf("the operator's line after 3 days: %d, want 1", len(e)) + } + }) + } +} diff --git a/hub/internal/api/end_notice_test.go b/hub/internal/api/end_notice_test.go index c897f792..1178770a 100644 --- a/hub/internal/api/end_notice_test.go +++ b/hub/internal/api/end_notice_test.go @@ -419,12 +419,12 @@ func TestNotice_StuckForThreeDays_TheOperatorGetsOneLine(t *testing.T) { t.Fatalf("the line was filed while the hold is on: %+v", e) } _ = os.Remove(marker) - r.h.endNotices("c1", now.Add(EndNoticeStuckAfter+3*time.Hour)) + r.h.endNotices("c1", now.Add(EndNoticeStuckAfter+6*time.Hour)) // past the wait after the third try (4 h) e := stuck() if len(e) != 1 || e[0].Severity != "warning" || !strings.Contains(e[0].Message, `"vikunja"`) || !strings.Contains(e[0].Message, "did not accept") || !strings.Contains(e[0].DetailsJSON, `"subject":"vikunja"`) { t.Fatalf("want ONE warning naming the app and the reason: %+v", e) } - for i := 5; i < 10; i++ { + for i := 7; i < 12; i++ { r.h.endNotices("c1", now.Add(EndNoticeStuckAfter+time.Duration(i)*time.Hour)) } if e := stuck(); len(e) != 1 { diff --git a/hub/internal/notify/end_notice.go b/hub/internal/notify/end_notice.go index 091faac3..31756292 100644 --- a/hub/internal/notify/end_notice.go +++ b/hub/internal/notify/end_notice.go @@ -26,7 +26,8 @@ import ( // is built so that a box can do nothing else with it: // - THE HUB WRITES IT. Subject and body come only from the hub's own bundle, with an app name the hub checked // (the shape of an app name, not reserved) and a day the hub computed from its own clock. Nothing of the box's. -// - AT MOST receiptsPerDay PER HOUSEHOLD IN 24 HOURS, counted in the hub's own sent log (it survives a restart). +// - AT MOST receiptsPerDay PER HOUSEHOLD IN 24 HOURS, counted in the hub's own sent log (it survives a restart) — +// every try that reached the mail service, `sent` or `failed`: a send that timed out here may have gone. // The limit DELAYS a notice — the record stays „not told” and the next report tries again — it never drops one. // - to the household's own notification address; never to the operator. // @@ -103,7 +104,7 @@ func (d *Dispatcher) SendEndNotice(customerID, app string, ends time.Time) (EndN return EndNoticeNoAddress, "the household has no notification address" } now := time.Now() - sentToday, cerr := d.store.CountCustomerSentSince(customerID, EndNoticeEvent, now.Add(-24*time.Hour)) + sentToday, cerr := d.store.CountCustomerTriesSince(customerID, EndNoticeEvent, now.Add(-24*time.Hour)) if cerr != nil { // A log that cannot be read is not „nothing sent”: no receipt goes, and the notice stays to be tried again. return EndNoticeLimit, "the hub's sent log could not be read" diff --git a/hub/internal/store/offsite_end_notice.go b/hub/internal/store/offsite_end_notice.go index 6f1e3b92..b704f1a6 100644 --- a/hub/internal/store/offsite_end_notice.go +++ b/hub/internal/store/offsite_end_notice.go @@ -299,11 +299,12 @@ func (s *Store) BackdateNotificationLogForTest(customerID string, by time.Durati return err } -// CountOperatorSentSince counts the operator mails of one type that the hub's own log says were SENT about a -// household since `since` (the daily limit of the operator's in-place notice). -func (s *Store) CountOperatorSentSince(customerID, eventType string, since time.Time) (int, error) { +// CountOperatorTriesSince counts the operator mails of one type about a household that REACHED the mail service +// since `since` — `sent` and `failed`, for the reason CountCustomerTriesSince gives (the daily limit of the +// operator's in-place notice). +func (s *Store) CountOperatorTriesSince(customerID, eventType string, since time.Time) (int, error) { var n int - err := s.db.QueryRow(`SELECT COUNT(*) FROM notification_log WHERE customer_id = ? AND event_type = ? AND channel = 'operator' AND status = 'sent' AND created_at >= ?`, + err := s.db.QueryRow(`SELECT COUNT(*) FROM notification_log WHERE customer_id = ? AND event_type = ? AND channel = 'operator' AND status IN ('sent', 'failed') AND created_at >= ?`, customerID, eventType, since.UTC().Format("2006-01-02 15:04:05")).Scan(&n) return n, err } diff --git a/hub/internal/store/offsite_ended.go b/hub/internal/store/offsite_ended.go index eb7d1b94..956dfa09 100644 --- a/hub/internal/store/offsite_ended.go +++ b/hub/internal/store/offsite_ended.go @@ -466,11 +466,23 @@ func (s *Store) BackdateLargeEndedRequestsForTest(customerID string, by time.Dur return nil } -// CountCustomerSentSince counts the mails of one type the hub's own log says were SENT to a household since `since` -// (the receipt's daily limit — it survives a hub restart, which a counter in memory would not). +// CountCustomerSentSince counts the mails of one type the hub's own log says were SENT to a household since `since`. func (s *Store) CountCustomerSentSince(customerID, eventType string, since time.Time) (int, error) { + return s.countCustomerLog(customerID, eventType, since, `status = 'sent'`) +} + +// CountCustomerTriesSince counts every try of one type that REACHED the mail service for a household since `since` +// — `sent` and `failed` (the notice's daily limit; it survives a hub restart, which a counter in memory would not). +// A try that failed is counted because „failed” is what the hub saw, not what the mail service did: a send that +// timed out on the hub's side may have gone (review of 2026-11-15, finding 2). A held mail never reached the +// service and is not counted. +func (s *Store) CountCustomerTriesSince(customerID, eventType string, since time.Time) (int, error) { + return s.countCustomerLog(customerID, eventType, since, `status IN ('sent', 'failed')`) +} + +func (s *Store) countCustomerLog(customerID, eventType string, since time.Time, status string) (int, error) { var n int - err := s.db.QueryRow(`SELECT COUNT(*) FROM notification_log WHERE customer_id = ? AND event_type = ? AND channel = 'customer' AND status = 'sent' AND created_at >= ?`, + err := s.db.QueryRow(`SELECT COUNT(*) FROM notification_log WHERE customer_id = ? AND event_type = ? AND channel = 'customer' AND `+status+` AND created_at >= ?`, customerID, eventType, since.UTC().Format("2006-01-02 15:04:05")).Scan(&n) return n, err }