hub (unreleased): the notice lock — no notice, no deletion; the hub sends the household's notice itself (decision 315, R-1374)

The hub agrees to remove a removed app's off-site copies only when its OWN record
(offsite_end_notices) says the household was told at least 30 days ago, by the hub's
clock — beside the 30 days of absence. The notice is sent from the box's report, at
every report until the mail service accepts it; the box's event mails nobody.
No address or a blocked household: the operator is mailed in its place and that mail
starts the 30 days. A notice that cannot go out: no agreement, the window's answer
names the reason, one line to the operator after 3 days.

This commit also supplies the files that handler.go (c3fa43a4) already calls
(store.NoteEndNotices, api.kickEndNotices) and the tests that replace
TestD84_Receipt_* (their files left the tree in b8cf6c24).

Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
admin committed 2026-11-15 16:17:44 +01:00
1 parent b8cf6c2417
commit 42ef25c23b
25 files changed
+2077 -184

No files matched your search

+4 -2
View File
@@ -77,8 +77,10 @@
| `offsitekeys.Registrar` (`Install` / `Confirm` / `Audit` / `OpenWindow` / `CloseWindow` / `MoveAside`) | hub/internal/offsitekeys/offsitekeys.go | `(ctx, Target, password, …)` | EVERY write to a sub-account's `.ssh/authorized_keys` and every repo move-aside | **The only writer of that file, and the only deleter on a sub-account (`DeleteSetAside`: `<repo>.orphaned-*` only, decision 74).** `read()` is read-only (R-827). Uses the provider's port-23 restricted shell (`dd of=` takes stdin, `mv` overwrites, `test` does NOT exist — measured); an unpinned line is a deletion route and is dropped on every install; the window line goes FIRST (first match wins). Never `rm`. |
| `offsitekeys.Service` (`RegisterKey`, `ConfirmKey`, `AuditAll`, `OpenWindowFor`, `CloseWindowFor`, `SweepExpiredWindows`) | hub/internal/offsitekeys/service.go | — | Binding the registrar to the store, descriptor and operator events | The box-facing API (`/api/v1/offsite/register-key…`) answers with NO credential — pinned by `TestOffsiteKeyEndpoints_AuthAndNoPasswordInAnyResponse`. |
| `offsitekeys` snapshot watch + `store.Get/SetOffsiteSnapshotListing` / `OffsiteRunTimesSince` (v0.158.0, R-895 slice 2) | hub/internal/offsitekeys/snapshot_watch.go, hub/internal/store/offsite_snapshots.go | — | Judging the 07:10 `snapshots/` listing against yesterday's (baseline in `hub_settings`) and the box's reported runs | Raises `offsite_snapshots_vanished` / `_unexplained` — operator-only and hub-origin (NOT in `allowedEventTypes`). The first listing is a baseline; a move-aside or RESET drops it. |
| `store.NoteAppList` / `AppPresences` + `offsitekeys.agreeEnded` (D72/D84, `07` §8.2f) | hub/internal/store/offsite_ended.go, hub/internal/offsitekeys/ended.go | `NoteAppList(customerID, deployed, claimed, now)`; `agreeEnded(now, presence, listing, haveListing, claims)` | The hub's OWN record of when an app left a box's app list (fed by `handleReport`), and the decision which snapshot ids of ended apps a window may remove outside its cap | **The box's dates decide nothing.** An id is agreed only when its app has been absent 30 days by the hub's clock AND the hub's own listing shows the file older than that absence. `WindowCredit.CapExplains` is the cap's part, `EndedIDs` the agreed ids — a reader that explains ids by name must not also spend `Explains`. **After the review of 2026-11-15:** one window agrees to at most the cap's size from the hub's OWN count (`endedCap`/`boundEndedToCap`); a request for more than half of the store waits a week — its first sight is the hub's own row (`store.NoteLargeEndedRequest`, key `largeSetKey` = a hash of the whole app set), never a search of text a box could write; `store.NoteAppListClaims` returns never-seen apps once for the operator's warning; a claimed name that is not an app name is written nowhere (`ledgerName`). |
| `notify.householdReceiptEvents` (D84) | hub/internal/notify/dispatcher.go | register | A mail that confirms the household's own press (one type: `offsite_end_scheduled`) | Bypasses the household's group switches and is keyed per press (details `subject`); the operator is not mailed. **Every new entry needs its own ruling** — pinned by `TestD84_Receipt_OpensNothingElse`. A receipt goes through `Dispatcher.processReceipt`, never `processCustomer`: the hub writes it from checked details (`parseReceipt`, `FormatReceiptEmail`), at most `receiptsPerDay` per household in 24 h (`store.CountCustomerSentSince`). |
| `store.NoteAppList` / `AppPresences` + `offsitekeys.agreeEnded` (D72/D84, `07` §8.2f) | hub/internal/store/offsite_ended.go, hub/internal/offsitekeys/ended.go | `NoteAppList(customerID, deployed, claimed, now)`; `agreeEnded(now, presence, notices, listing, haveListing, claims)` | The hub's OWN record of when an app left a box's app list (fed by `handleReport`), and the decision which snapshot ids of ended apps a window may remove outside its cap | **The box's dates decide nothing.** An id is agreed only when its app has been absent 30 days by the hub's clock AND the hub's own listing shows the file older than that absence. `WindowCredit.CapExplains` is the cap's part, `EndedIDs` the agreed ids — a reader that explains ids by name must not also spend `Explains`. **After the review of 2026-11-15:** one window agrees to at most the cap's size from the hub's OWN count (`endedCap`/`boundEndedToCap`); a request for more than half of the store waits a week — its first sight is the hub's own row (`store.NoteLargeEndedRequest`, key `largeSetKey` = a hash of the whole app set), never a search of text a box could write; `store.NoteAppListClaims` returns never-seen apps once for the operator's warning; a claimed name that is not an app name is written nowhere (`ledgerName`). |
| `notify.householdReceiptEvents` + `Dispatcher.SendEndNotice` (D84; the notice lock, `09` §3 decision 315) | hub/internal/notify/end_notice.go | register; `SendEndNotice(customerID, app, ends) (EndNoticeOutcome, why)` | The mail that tells the household about its own press (one type: `offsite_end_scheduled`). **The box's event mails nobody**: the hub sends the notice from its own state (`api.endNotices`, at every report, until the mail service accepts it) | Bypasses the household's group switches; the hub writes it from a checked app name and its OWN day. `SendEndNotice` keeps no state and tries once: only `EndNoticeSent` tells the household; the daily limit (5 per household, counted in the sent log) answers `EndNoticeLimit` — the caller tries again, it never drops. **Every new entry needs its own ruling** — pinned by `TestD84_Receipt_OpensNothingElse`, `TestNotice_*`. |
| `store.NoteEndNotices` / `EndNotices` / `MarkEndNoticeTold` + `api.endNotices` (the notice lock, `07` §8.2f) | hub/internal/store/offsite_end_notice.go, hub/internal/api/end_notice.go | `NoteEndNotices(customerID, deployed, claims, tierOn, now)`; `EndNotices(customerID)`; `MarkEndNoticeTold(customerID, app, press, to, now)` | The hub's OWN record that a household (or the operator in its place) was told that an app's off-site copies will be deleted. `offsitekeys.agreeEnded` agrees only when it says „told” 30 days ago | **Written only by the hub, only after the mail service accepted the mail; never derived from the sent log or from a box's date.** `press` (the box's `removed_at`) is a key only: another value VOIDS the notice. An install, a take-back (a report that shows the tier ON and no record) and a household delete remove the row. A new condition of a deletion goes HERE, not into `notification_log`. |
| `Dispatcher.SendOperatorNow` | hub/internal/notify/end_notice.go | `SendOperatorNow(customerID, eventType, severity, message, detailsJSON) error` | An operator mail whose DELIVERY is a condition of something else: it answers only when the mail service has (nil = accepted) | Operator-only types only; no cooldown and no background retry — the caller keeps its own „told once” record and tries again. For every other operator mail use `ProcessEvent`. |
| `accesslog.New` / `Middleware` / `Mark` (v0.158.0, R-1140) | hub/internal/accesslog/accesslog.go | `Mark(ctx, kind)` | The hub's own access log (`<data_dir>/access-log/`, 90 days) and marking the identity KIND an auth check established | Never log a value: the path only (no query; `/bind/<token>` and 32+-char segments redacted). A new auth path calls `Mark` with its kind. A logging failure never fails a request. |
| the timed SQLite driver `sqlite-timed` + `SetSlowStatementThreshold` (v0.178.0, R-1346) | hub/internal/store/timeddriver.go | `store.New` opens it — nothing to call | Names a statement over 1 s: `[WARN] slow statement: <ms> (over 1s): <SQL>` | The SQL text only, never a bound value. Put a heavy statement's SQL in a const and pin its plan (`r1346_speed.go`, `TestR1346_HeavyStatementsUseTheirIndex`): on the hub's network disk a whole-table read is seconds to minutes. |
| `(*Store).SaveOneTimeSecret` / `OffsitePassword` / `SealLegacyOffsiteSecrets` | hub/internal/store/offsite_seal.go | — | Storing / reading the sub-account password | **Sealed AES-256-GCM; no key → refused (fail-closed).** Under `go test` every store gets a fixed key (`testing.Testing()`); production needs `OFFSITE_SECRET_KEY`. Never serve the value to a box. |
@@ -0,0 +1,40 @@
== MUTATION: no notice lock: the absence alone decides (the first build)
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_AgreeEnded_NoNoticeNoDeletion (0.00s)
notice_lock_test.go:59: agreed ids = [a1b2c685], want [] (refused: [])
notice_lock_test.go:59: agreed ids = [a1b2c685], want [] (refused: [])
notice_lock_test.go:59: agreed ids = [a1b2c685], want [] (refused: [])
== MUTATION: a record that nobody was told about is enough
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_AgreeEnded_NoNoticeNoDeletion (0.00s)
notice_lock_test.go:59: agreed ids = [a1b2c685], want [] (refused: [])
notice_lock_test.go:59: agreed ids = [a1b2c685], want [] (refused: [])
notice_lock_test.go:59: agreed ids = [a1b2c685], want [] (refused: [])
== MUTATION: told, however recently, is enough
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_AgreeEnded_NoNoticeNoDeletion (0.00s)
notice_lock_test.go:59: agreed ids = [a1b2c685], want [] (refused: [])
notice_lock_test.go:59: agreed ids = [a1b2c685], want [] (refused: [])
notice_lock_test.go:59: agreed ids = [a1b2c685], want [] (refused: [])
== MUTATION: the told time counts 29 days
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_AgreeEnded_NoNoticeNoDeletion (0.00s)
notice_lock_test.go:59: agreed ids = [a1b2c685], want [] (refused: [])
--- FAIL: TestNotice_Window_TheNoticeRecordDecides (0.14s)
notice_lock_test.go:114: told 29 days ago: agreed [a1b2c684 a1b2c685], refused []
== MUTATION: an app with no notice record at all is agreed
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_AgreeEnded_NoNoticeNoDeletion (0.00s)
notice_lock_test.go:59: agreed ids = [a1b2c685], want [] (refused: [])
notice_lock_test.go:59: agreed ids = [a1b2c685 a1b2c686], want [a1b2c685] (refused: [])
--- FAIL: TestNotice_Window_TheNoticeRecordDecides (0.05s)
== MUTATION: a box that asks for an app it never named in a report is refused quietly
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_Window_TheNoticeRecordDecides (0.05s)
notice_lock_test.go:86: the operator must hear of a box that asks for an app it never named in a report: []
== MUTATION: the notice is read IN PLACE of the absence
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_AgreeEnded_NoNoticeNoDeletion (0.00s)
notice_lock_test.go:59: agreed ids = [a1b2c685], want [] (refused: [])
7 mutations, 0 not a red-proof
@@ -0,0 +1,54 @@
== MUTATION: a report never starts the walk
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_AReportNamesAnApp_TheHubMailsTheHouseholdOncePerPress (0.05s)
end_notice_test.go:162: the household got 0 notice(s) after the first report that names the app, want 1: []
--- FAIL: TestNotice_TwoAppsInOneMinute_TwoNotices (0.05s)
end_notice_test.go:196: two apps: []
== MUTATION: the next try does not wait
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_AFailedSendIsTriedAgainUntilItHasGone (0.05s)
end_notice_test.go:266: a second try ten minutes after a failed one: 1 mail(s)
== MUTATION: a failed notice is never tried again
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_AFailedSendIsTriedAgainUntilItHasGone (0.07s)
end_notice_test.go:272: the try an hour later must send and set the told time to THAT try (2026-11-15T16:10:54Z): mails=0 {App:vikunja Press:2026-11-15T10:00:00Z FirstNamed:2026-11-15 15:09:54 +0000 UTC ToldAt:0001-01-01 00:00:00
--- FAIL: TestNotice_TheMailHoldTellsNobody (0.04s)
end_notice_test.go:295: after the release: {App:vikunja Press:2026-11-15T10:00:00Z FirstNamed:2026-11-15 15:09:54 +0000 UTC ToldAt:0001-01-01 00:00:00 +0000 UTC ToldTo: Tries:1 LastTry:2026-11-15 15:09:54 +0000 UTC LastReason:the
== MUTATION: the operator is not told in place of a household with no address
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_NoAddress_TheOperatorIsToldInItsPlace_AndAnAddressLaterRestarts (0.05s)
end_notice_test.go:346: the operator must be told once, naming the household, the app and why: []
--- FAIL: TestNotice_NoAddress_AndTheOperatorsMailFails_NobodyWasTold (0.05s)
end_notice_test.go:392: no address and the operator's mail failed: {App:vikunja Press:2026-11-15T10:00:00Z FirstNamed:2026-11-15 15:09:59 +0000 UTC ToldAt:0001-01-01 00:00:00 +0000 UTC ToldTo: Tries:0 LastTry:0001-01-01 00:00:00 +
== MUTATION: the operator's notice counts although its mail failed
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_NoAddress_AndTheOperatorsMailFails_NobodyWasTold (0.05s)
end_notice_test.go:392: no address and the operator's mail failed: {App:vikunja Press:2026-11-15T10:00:00Z FirstNamed:2026-11-15 15:10:03 +0000 UTC ToldAt:2026-11-15 15:10:03 +0000 UTC ToldTo:operator Tries:1 LastTry:2026-11-15 15
== MUTATION: an address added later changes nothing (the operator's notice stays the one that counts)
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_NoAddress_TheOperatorIsToldInItsPlace_AndAnAddressLaterRestarts (0.06s)
end_notice_test.go:372: an address added after the press: the household must be told and the 30 days start again: mails=[{to:operator@felhom.eu subject:[Felhom] ⚠️ c1: offsite_end_notice_operator body:Customer: c1
== MUTATION: the operator's stuck line is filed at once
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_StuckForThreeDays_TheOperatorGetsOneLine (0.05s)
end_notice_test.go:409: told the operator after two days: [{ID:1 CustomerID:c1 EventType:offsite_end_notice_stuck Severity:warning Message:Off-site: the household c1 removed the app "vikunja" with „delete backup data” 0 days ago,
== MUTATION: the operator's stuck line is never filed
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_StuckForThreeDays_TheOperatorGetsOneLine (0.05s)
end_notice_test.go:425: want ONE warning naming the app and the reason: []
== MUTATION: the operator's stuck line is repeated
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_StuckForThreeDays_TheOperatorGetsOneLine (0.05s)
end_notice_test.go:431: the line was repeated: 6
== MUTATION: the stuck line is filed while the mail hold is on
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_StuckForThreeDays_TheOperatorGetsOneLine (0.05s)
end_notice_test.go:419: the line was filed while the hold is on: [{ID:1 CustomerID:c1 EventType:offsite_end_notice_stuck Severity:warning Message:Off-site: the household c1 removed the app "vikunja" with „delete backup data” 3 day
== MUTATION: the told time is the box's press, not the hub's clock
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_AReportNamesAnApp_TheHubMailsTheHouseholdOncePerPress (0.05s)
end_notice_test.go:173: the hub's own record does not say the household was told now: {App:vikunja Press:2026-11-15T10:00:00Z FirstNamed:2026-11-15 15:10:23 +0000 UTC ToldAt:2026-11-15 10:00:00 +0000 UTC ToldTo:household Tries:1 L
--- FAIL: TestNotice_AFailedSendIsTriedAgainUntilItHasGone (0.06s)
end_notice_test.go:272: the try an hour later must send and set the told time to THAT try (2026-11-15T16:11:23Z): mails=1 {App:vikunja Press:2026-11-15T10:00:00Z FirstNamed:2026-11-15 15:10:23 +0000 UTC ToldAt:2026-11-15 10:00:00
11 mutations, 0 not a red-proof
@@ -0,0 +1,45 @@
# Two mutants of the first run did not compile (an unused variable) and proved nothing; they were rewritten and the whole file was run again. This is the second run.
== MUTATION: a failed send is reported as sent
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_AFailedSendIsTriedAgainUntilItHasGone (0.04s)
end_notice_test.go:260: after a failed send: {App:vikunja Press:2026-11-15T10:00:00Z FirstNamed:2026-11-15 15:11:08 +0000 UTC ToldAt:2026-11-15 15:11:08 +0000 UTC ToldTo:household Tries:1 LastTry:2026-11-15 15:11:08 +0000 UTC Last
--- FAIL: TestNotice_StuckForThreeDays_TheOperatorGetsOneLine (0.05s)
end_notice_test.go:425: want ONE warning naming the app and the reason: []
== MUTATION: a held mail is reported as sent
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_TheMailHoldTellsNobody (0.04s)
end_notice_test.go:288: under the hold: {App:vikunja Press:2026-11-15T10:00:00Z FirstNamed:2026-11-15 15:11:12 +0000 UTC ToldAt:2026-11-15 15:11:12 +0000 UTC ToldTo:household Tries:1 LastTry:2026-11-15 15:11:12 +0000 UTC LastReaso
--- FAIL: TestNotice_StuckForThreeDays_TheOperatorGetsOneLine (0.06s)
end_notice_test.go:425: want ONE warning naming the app and the reason: []
== MUTATION: the daily limit is not read
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_TheDailyLimitDelaysAndNeverDrops (0.08s)
end_notice_test.go:309: seven presses: 7 notices today, want the daily limit 5
--- FAIL: TestNotice_TheDailyLimitDelaysAndSurvivesARestart (0.05s)
end_notice_test.go:157: the sixth notice: outcome sent (""), want limit
== MUTATION: a blocked household is mailed
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_NoAddress_TheOperatorIsToldInItsPlace_AndAnAddressLaterRestarts (0.12s)
end_notice_test.go:381: a blocked household: record {App:vikunja Press:2026-11-15T10:00:00Z FirstNamed:2026-11-15 15:11:22 +0000 UTC ToldAt:2026-11-15 15:11:22 +0000 UTC ToldTo:household Tries:1 LastTry:2026-11-15 15:11:22 +0000 U
--- FAIL: TestNotice_NoAddressOrBlocked_SaysSoAndMailsNobody (0.08s)
end_notice_test.go:137: blocked: outcome sent, 1 mail(s)
== MUTATION: any name is mailed
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_OnlyAnAppNameIsMailed (0.06s)
end_notice_test.go:212: "": outcome sent, 1 mail(s)
== MUTATION: SendOperatorNow takes any type
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_SendOperatorNow (0.05s)
end_notice_test.go:246: a type that is not operator-only was accepted
== MUTATION: SendOperatorNow answers nil for a failed send
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_SendOperatorNow (0.05s)
end_notice_test.go:250: a failed send answered nil — the caller would start the 30 days on a mail that never went
7 mutations, 0 not a red-proof
== MUTATION: the box's event is routed like any other event again
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_TheBoxesEventMailsNobody (0.06s)
end_notice_test.go:113: the box's event caused 1 mail(s); first: {to:operator@felhom.eu subject:[Felhom] ⚠️ c1: offsite_end_scheduled body:Customer: c1
1 mutations, 0 not a red-proof
@@ -0,0 +1,47 @@
== MUTATION: an install does not void the notice
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_Record_WhatVoidsIt (0.35s)
offsite_end_notice_test.go:113: the notice is told, want gone
== MUTATION: another press keeps the old notice
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_Record_WhatVoidsIt (0.35s)
offsite_end_notice_test.go:113: the notice is told, want fresh
offsite_end_notice_test.go:113: the notice is told, want fresh
== MUTATION: a take-back does not void the notice
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_Record_WhatVoidsIt (0.35s)
offsite_end_notice_test.go:113: the notice is told, want gone
== MUTATION: a report that does not show the tier voids the notice (its silence read as a take-back)
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_Record_WhatVoidsIt (0.35s)
offsite_end_notice_test.go:113: the notice is gone, want told
== MUTATION: a told mark is written whatever the press
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_Record_OnePressOneRow (0.05s)
offsite_end_notice_test.go:47: a told mark for ANOTHER press was written (ok=true err=<nil>)
== MUTATION: a recipient the hub never writes counts as told
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_Record_IsBoundedAndFailsClosed (0.05s)
offsite_end_notice_test.go:160: an unreadable record counts as told: {App:app-00 Press: FirstNamed:2026-11-15 10:00:00 +0000 UTC ToldAt:2020-01-01 00:00:00 +0000 UTC ToldTo:box Tries:0 LastTry:0001-01-01 00:00:00 +0000 UTC LastRea
== MUTATION: no row limit per household
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_Record_IsBoundedAndFailsClosed (0.05s)
offsite_end_notice_test.go:136: rows = 80, want the limit 50
== MUTATION: a name that is not an app name is stored
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_Record_IsBoundedAndFailsClosed (0.05s)
offsite_end_notice_test.go:140: a name that is not an app name was stored: "<b>x</b>"
8 mutations, 0 not a red-proof
== MUTATION: the notice record is not purged with the household
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_Record_GoesWithTheHousehold (0.05s)
offsite_end_notice_test.go:172: the residue count does not see the notice record: &{Reports:0 AppTelemetry:0 AppLogTails:0 LogTailRequests:0 NotificationPrefs:0 SelfBindTokens:0 ApplianceRegistrations:0 AppLogIssueRefs:0 OffsiteKe
1 mutations, 0 not a red-proof
== MUTATION: the notice record is not in the orphan sweep
RED on the mutant, GREEN on the real file
--- FAIL: TestNotice_Record_GoesWithTheHousehold (0.05s)
offsite_end_notice_test.go:184: orphan sweep: map[] <nil>
1 mutations, 0 not a red-proof
+20
View File
@@ -1,5 +1,25 @@
## Unreleased (after v0.182.0)
- **The notice lock (2026-11-15 evening, `09` §3 decision 315, R-1374; `07` §8.2f): no notice, no deletion.** The
household's mail about „delete backup data” was sent once, from the box's event — a lost event, a failed send or
the daily limit, and the hub still agreed 30 days later. Now **the hub sends the notice itself, from its own
state**: every controller report feeds a new table, `offsite_end_notices` (`store.NoteEndNotices` — one row per app
the report names in a pending end record), and `api.endNotices` mails each row's household
(`notify.SendEndNotice`) at every report until the mail service accepts the mail; only then does the row say
„told”, by the hub's clock. **`offsitekeys.agreeEnded` agrees to an app only when that row says „told” at least 30
days ago — beside the 30 days of absence.** The box's event `offsite_end_scheduled` is recorded and mails nobody.
The day in the mail is the hub's own (its clock + 30 days). A household with no address, or a blocked one: the
operator is mailed in its place, once per app (`offsite_end_notice_operator`, sent with `SendOperatorNow`, which
answers only when the mail service has), and that mail starts the 30 days; an address added later gives the
household its own mail and starts them again *(decided by CC unattended — the operator may reverse it)*. An
install, a take-back (a report that shows the tier on and no record) and another press void the notice; a
database put back from a copy that lacks the row tells again and waits again. The daily limit (5) delays a notice
and never drops one. The mail hold or a failed send: nobody is told, the window's answer names the reason, the
next try is an hour later, and after **3 days** the operator gets one line (`offsite_end_notice_stuck`). A box
that asks a window for an app it never named in a report is refused and told to the operator
(`offsite_ended_app_refused`). No controller change: an older controller sends no end records and nothing of it is
removed, as before. Tests `TestNotice_*` (store 4, offsitekeys 2, api 13, notify 8); 36 mutations seen red (two
redone: they did not compile). `TestD84_Receipt_*` (the event-driven mail) are replaced by them.
- **D83, the hub's half (2026-11-15 evening, R-1398 part 2; `16` §14.5): the hub shows which boxes were accepted on a
small system disk.** `POST /host-enroll` reads `small_disk` (only the JSON `true`) and, when a later installer sends
it, `system_disk_bytes`; `store.NoteSmallDisk` / `SmallDiskOf` keep it as one of the marks that name the box
+196
View File
@@ -0,0 +1,196 @@
package api
import (
"encoding/json"
"fmt"
"sort"
"sync"
"time"
"gitea.dooplex.hu/admin/felhom-hub/internal/notify"
"gitea.dooplex.hu/admin/felhom-hub/internal/offsitekeys"
"gitea.dooplex.hu/admin/felhom-hub/internal/store"
)
// THE NOTICE LOCK (`09` §3 decision 315; `07-backup-architecture.md` §8.2f „No notice, no deletion”) — the hub sends
// the household's notice ITSELF, from its own state, until it has gone.
//
// Every controller report feeds the hub's record (store.NoteEndNotices: which apps the box names in a pending end
// record). After that, endNotices walks the household's rows and, for each one whose household has not been told:
//
// - tries the household's mail (notify.SendEndNotice). Sent → the record says „told” at the hub's time, and the
// hub agrees to remove that app's copies no earlier than 30 days after it (offsitekeys.agreeEnded).
// - no address, or a blocked household → the OPERATOR is mailed in its place, once per app
// (`offsite_end_notice_operator`), and that mail starts the 30 days. When an address appears later, the household
// gets its own mail and the 30 days start again from it.
// - the daily limit, the mail hold, a send that fails → nothing is told; the reason is written on the row (the
// window's answer names it) and the next try is at a report at least endNoticeRetryGap later.
// - a row that has not been told for EndNoticeStuckAfter → one line to the operator (`offsite_end_notice_stuck`).
//
// Nothing here reads a date a box sent, and no event of a box starts or stops anything.
const (
// endNoticeRetryGap: after a try that told nobody, the next try waits this long (a report comes every 15 minutes;
// a mail service that is down is not asked 96 times a day per app).
endNoticeRetryGap = time.Hour
// EndNoticeStuckAfter: how long a notice may stay unsent before the operator gets one line. THREE DAYS: longer
// than any outage of the mail service the hub has had and than the hold on a restored copy is meant to last, and
// longer than the daily limit delays an honest household (5 notices a day: a household that removes up to 15
// apps at once is told about all of them inside it) — so the line means „this will not heal by itself”. Short
// enough that a deletion the household asked for has lost three days, not a month, when the operator hears.
EndNoticeStuckAfter = 3 * 24 * time.Hour
)
// endNoticeSpawn runs the sends off the report's request (a send can wait 10 s on the mail service). A seam: tests
// run it in line.
var endNoticeSpawn = func(f func()) { go f() }
// endNoticeBusy: one walk per household at a time — a second report that arrives while a walk is still sending does
// not send the same notice again; its rows are picked up by the next report.
var endNoticeBusy sync.Map // customerID → *sync.Mutex
// kickEndNotices starts a walk when the household has a row that still needs one.
func (h *Handler) kickEndNotices(customerID string) {
ns, err := h.store.EndNotices(customerID)
if err != nil {
h.logger.Printf("[WARN] [end-notice] %s: the hub's own notice record could not be read: %v", customerID, err)
return
}
work := false
for _, n := range ns {
if n.ToldTo != store.EndNoticeHousehold {
work = true
}
}
if work {
endNoticeSpawn(func() { h.endNotices(customerID, time.Now()) })
}
}
// endNotices is one walk over a household's rows (see the header). now is the HUB's clock.
func (h *Handler) endNotices(customerID string, now time.Time) {
mu, _ := endNoticeBusy.LoadOrStore(customerID, &sync.Mutex{})
if !mu.(*sync.Mutex).TryLock() {
return
}
defer mu.(*sync.Mutex).Unlock()
ns, err := h.store.EndNotices(customerID)
if err != nil {
h.logger.Printf("[WARN] [end-notice] %s: the hub's own notice record could not be read: %v", customerID, err)
return
}
apps := make([]string, 0, len(ns))
for app := range ns {
apps = append(apps, app)
}
sort.Strings(apps)
for _, app := range apps {
n := ns[app]
if n.ToldTo == store.EndNoticeHousehold {
continue
}
if n.LastReason != "" && !n.LastTry.IsZero() && now.Sub(n.LastTry) < endNoticeRetryGap {
h.endNoticeStuck(customerID, n, now)
continue
}
outcome, why := notify.EndNoticeFailed, "the hub has no mail service configured"
if h.dispatcher != nil {
// The day in the mail is the hub's own: its clock now + 30 days. Never the box's date.
outcome, why = h.dispatcher.SendEndNotice(customerID, app, now.Add(offsitekeys.EndedAppAfter))
}
switch outcome {
case notify.EndNoticeSent:
ok, merr := h.store.MarkEndNoticeTold(customerID, app, n.Press, store.EndNoticeHousehold, now)
if merr != nil || !ok {
// The mail went and the record does not say so: the household is mailed again at a later report and
// the 30 days start then. The safe direction (no deletion on a notice the hub cannot show).
h.logger.Printf("[WARN] [end-notice] %s/%s: the household's notice was sent, but the hub's record was NOT written (matched=%v, err=%v) — it will be sent again", customerID, app, ok, merr)
continue
}
n.ToldAt, n.ToldTo = now, store.EndNoticeHousehold
h.logger.Printf("[INFO] [end-notice] %s/%s: the household was told; the hub agrees to remove this app's off-site copies no earlier than %s",
customerID, app, now.Add(offsitekeys.EndedAppAfter).UTC().Format("2006-01-02"))
case notify.EndNoticeNoAddress:
if n.Told() {
continue // the operator was told in its place; the household still has no address
}
if terr := h.endNoticeToOperator(customerID, n, why, now); terr != nil {
why = why + "; and the operator could not be told in its place (" + terr.Error() + ")"
h.endNoticeTried(customerID, &n, why, now)
} else {
n.ToldAt, n.ToldTo = now, store.EndNoticeOperator
}
default: // the daily limit, the hold, a failed send
h.endNoticeTried(customerID, &n, why, now)
}
h.endNoticeStuck(customerID, n, now)
}
}
func (h *Handler) endNoticeTried(customerID string, n *store.EndNotice, why string, now time.Time) {
if err := h.store.MarkEndNoticeTried(customerID, n.App, n.Press, why, now); err != nil {
h.logger.Printf("[WARN] [end-notice] %s/%s: the try could not be recorded: %v", customerID, n.App, err)
}
n.LastReason, n.LastTry = why, now
h.logger.Printf("[WARN] [end-notice] %s/%s: nobody was told (%s) — nothing of this app is removed; tried again at a later report", customerID, n.App, why)
}
// endNoticeToOperator mails the operator in place of a household the hub cannot mail, and only when the mail service
// accepted that mail does the record say „told” (to the operator): that mail starts the app's 30 days.
// DECIDED BY CC UNATTENDED 2026-11-15 (the brief's A.3) — the operator may reverse it: `07` §8.2f.
func (h *Handler) endNoticeToOperator(customerID string, n store.EndNotice, why string, now time.Time) error {
if h.dispatcher == nil {
return fmt.Errorf("the hub has no mail service configured")
}
day := now.Add(offsitekeys.EndedAppAfter).UTC().Format("2006-01-02")
msg := fmt.Sprintf("Off-site: the household %s removed the app %q with „delete backup data”, and the hub could not tell the household: %s. "+
"This mail is the notice in its place. The hub counts the 30 days from now: from %s a clean-up window may remove that app's off-site copies. "+
"Tell the household. If the household did not ask for this, it can take it back on its Backups page (“Keep it after all”) and must change its password. "+
"When the household gets a notification address, the hub mails the household itself and the 30 days start again from that mail.",
customerID, n.App, why, day)
details, _ := json.Marshal(map[string]any{"subject": n.App, "app": n.App, "agrees_from": day})
if err := h.dispatcher.SendOperatorNow(customerID, offsitekeys.EventEndNoticeOperator, "warning", msg, string(details)); err != nil {
return err
}
if _, err := h.store.SaveEvent(customerID, offsitekeys.EventEndNoticeOperator, "warning", msg, string(details), "hub"); err != nil {
h.logger.Printf("[WARN] [end-notice] %s event save FAILED for %s (the mail went): %v", offsitekeys.EventEndNoticeOperator, customerID, err)
}
ok, err := h.store.MarkEndNoticeTold(customerID, n.App, n.Press, store.EndNoticeOperator, now)
if err != nil || !ok {
return fmt.Errorf("the operator's mail went, but the hub's record was not written (matched=%v, err=%v)", ok, err)
}
h.logger.Printf("[WARN] [end-notice] %s/%s: the household could not be told (%s) — the OPERATOR was told in its place; the hub agrees no earlier than %s", customerID, n.App, why, day)
return nil
}
// endNoticeStuck files the operator's one line for a notice that has told nobody for EndNoticeStuckAfter. Not while
// the mail hold is on (a held hub mails nobody; the line is filed at the first report after the release).
func (h *Handler) endNoticeStuck(customerID string, n store.EndNotice, now time.Time) {
if n.Told() || !n.StuckTold.IsZero() || n.FirstNamed.IsZero() || now.Sub(n.FirstNamed) < EndNoticeStuckAfter {
return
}
if h.dispatcher != nil && h.dispatcher.MailHeld() {
return
}
why := n.LastReason
if why == "" {
why = "no try has been recorded"
}
msg := fmt.Sprintf("Off-site: the household %s removed the app %q with „delete backup data” %d days ago, and the notice about it has not gone out: %s. "+
"NOTHING of that app's off-site copies is removed until the notice has gone and 30 days have passed after it. "+
"The household asked for this deletion and it is waiting: look at why the mail does not go. This line is sent once per app.",
customerID, n.App, int(now.Sub(n.FirstNamed).Hours()/24), why)
details, _ := json.Marshal(map[string]any{"subject": n.App, "app": n.App, "reason": why})
if _, err := h.store.SaveEvent(customerID, offsitekeys.EventEndNoticeStuck, "warning", msg, string(details), "hub"); err != nil {
h.logger.Printf("[WARN] [end-notice] %s event save FAILED for %s: %v", offsitekeys.EventEndNoticeStuck, customerID, err)
return // the audit row first: a mail without its event row lies
}
if err := h.store.MarkEndNoticeStuckTold(customerID, n.App, n.Press, now); err != nil {
h.logger.Printf("[WARN] [end-notice] %s/%s: the operator's line could not be marked as filed: %v", customerID, n.App, err)
}
h.logger.Printf("[WARN] [end-notice] %s/%s: the notice has told nobody for %d days (%s) — operator told once", customerID, n.App, int(now.Sub(n.FirstNamed).Hours()/24), why)
if h.dispatcher != nil {
h.dispatcher.ProcessEvent(customerID, offsitekeys.EventEndNoticeStuck, "warning", msg, string(details), "hub")
}
}
+509
View File
@@ -0,0 +1,509 @@
package api
import (
"errors"
"fmt"
"io"
"log"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
"gitea.dooplex.hu/admin/felhom-hub/internal/mailhold"
"gitea.dooplex.hu/admin/felhom-hub/internal/notify"
"gitea.dooplex.hu/admin/felhom-hub/internal/offsitekeys"
"gitea.dooplex.hu/admin/felhom-hub/internal/store"
)
// THE NOTICE LOCK (`09` §3 decision 315; `07-backup-architecture.md` §8.2f „No notice, no deletion”): the hub sends
// the household's notice itself, from the box's REPORT, until it has gone — and its own record says „told” only when
// the mail service accepted the mail. Every test here drives the real report endpoint; no test posts the box's event.
// RED-PROOFS: documentation/audits/build-release-4b-2026-11-15/A/redproof-notice-hub.txt.
type noticeMail struct{ to, subject, body string }
type noticeRig struct {
t *testing.T
h *Handler
st *store.Store
d *notify.Dispatcher
mu sync.Mutex
mails []noticeMail
fail error // when set, every send fails with it
}
// newNoticeRig: a household c1 with a notification address (email "" = none), a dispatcher whose sends are recorded,
// and the sends run in line with the report.
func newNoticeRig(t *testing.T, email string) *noticeRig {
t.Helper()
h, st, _ := newTestHandler(t)
if err := st.SaveCustomerConfig(&store.CustomerConfig{CustomerID: "c1", APIKey: "ckey", RetrievalPassword: "pp"}); err != nil {
t.Fatal(err)
}
if email != "" {
if err := st.SaveNotificationPrefs("c1", email, []string{}, 6); err != nil { // every switch off
t.Fatal(err)
}
}
r := &noticeRig{t: t, h: h, st: st}
r.d = notify.NewDispatcher(st, "test-key-not-used", "from@felhom.eu", "operator@felhom.eu", true, log.New(io.Discard, "", 0))
r.d.SetSendFnForTest(func(to, subject, body string, _ map[string]string) error {
r.mu.Lock()
defer r.mu.Unlock()
if r.fail != nil {
return r.fail
}
r.mails = append(r.mails, noticeMail{to, subject, body})
return nil
})
h.SetDispatcher(r.d)
old := endNoticeSpawn
endNoticeSpawn = func(f func()) { f() }
t.Cleanup(func() { endNoticeSpawn = old })
// vikunja and kimai were seen installed: the never-seen warning (offsite_end_claim_unseen) is not this file's subject
r.report(`{"customer_id":"c1","stacks":{"deployed":["immich","vikunja","kimai"]},"offsite":{"enabled":true}}`)
return r
}
// all is a copy of every mail so far (the never-seen warning of another feature is sent off the request).
func (r *noticeRig) all() []noticeMail {
r.mu.Lock()
defer r.mu.Unlock()
return append([]noticeMail(nil), r.mails...)
}
// opNotice: the operator's mails of one event type.
func (r *noticeRig) opNotice(typ string) []noticeMail {
var out []noticeMail
for _, m := range r.to(operator) {
if strings.Contains(m.subject, typ) {
out = append(out, m)
}
}
return out
}
func (r *noticeRig) report(body string) {
r.t.Helper()
req := httptest.NewRequest(http.MethodPost, "/api/v1/report", strings.NewReader(body))
req.Header.Set("Authorization", "Bearer ckey")
rr := httptest.NewRecorder()
r.h.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
r.t.Fatalf("report → %d (%s)", rr.Code, rr.Body.String())
}
}
// to counts the mails that went to one address.
func (r *noticeRig) to(addr string) []noticeMail {
r.mu.Lock()
defer r.mu.Unlock()
var out []noticeMail
for _, m := range r.mails {
if m.to == addr {
out = append(out, m)
}
}
return out
}
func (r *noticeRig) notice(app string) (store.EndNotice, bool) {
r.t.Helper()
ns, err := r.st.EndNotices("c1")
if err != nil {
r.t.Fatal(err)
}
n, ok := ns[app]
return n, ok
}
func (r *noticeRig) events(typ string) []store.Event {
r.t.Helper()
evs, err := r.st.GetEventsByType("c1", typ, time.Now().Add(-time.Hour))
if err != nil {
r.t.Fatal(err)
}
return evs
}
const (
household = "customer@example.com"
operator = "operator@felhom.eu"
// installed: immich runs; nothing is pending
repInstalled = `{"customer_id":"c1","stacks":{"deployed":["immich","vikunja"]},"offsite":{"enabled":true}}`
// the household removed vikunja with „delete backup data”
repPressed = `{"customer_id":"c1","stacks":{"deployed":["immich"]},"offsite":{"enabled":true,"ended_apps":[{"app":"vikunja","removed_at":"2026-11-15T10:00:00Z","ends_at":"1999-01-31T10:00:00Z"}]}}`
// the tier is on and no record is pending: a take-back (or the window has run)
repKept = `{"customer_id":"c1","stacks":{"deployed":["immich"]},"offsite":{"enabled":true}}`
)
func pressed(app, removedAt string) string {
return fmt.Sprintf(`{"customer_id":"c1","stacks":{"deployed":["immich"]},"offsite":{"enabled":true,"ended_apps":[{"app":%q,"removed_at":%q,"ends_at":"2026-12-15T10:00:00Z"}]}}`, app, removedAt)
}
// 1. THE HUB SENDS THE NOTICE ITSELF — the box's event never arrives in this test. One press is one notice, however
// many reports name it; the record says „told” at the hub's time; the mail's day is the HUB's (its clock + 30 days),
// not the box's `ends_at` (a day in 1999 here).
func TestNotice_AReportNamesAnApp_TheHubMailsTheHouseholdOncePerPress(t *testing.T) {
r := newNoticeRig(t, household)
r.report(repInstalled)
if len(r.all()) != 0 {
t.Fatalf("an installed app caused %d mail(s)", len(r.all()))
}
before := time.Now().Add(-time.Second)
r.report(repPressed)
got := r.to(household)
if len(got) != 1 {
t.Fatalf("the household got %d notice(s) after the first report that names the app, want 1: %+v", len(got), r.all())
}
hubDay := time.Now().Add(offsitekeys.EndedAppAfter)
if !strings.Contains(got[0].subject, "vikunja: ") || strings.Contains(got[0].subject+got[0].body, "1999") {
t.Fatalf("the notice must name the app and the HUB's day, never the box's ends_at:\n%s\n%s", got[0].subject, got[0].body)
}
if y := fmt.Sprintf("Törlésük napja: %d.", hubDay.Year()); !strings.Contains(got[0].body, y) {
t.Fatalf("the mail's day is not the hub's clock + 30 days (%s): %s", y, got[0].body)
}
n, ok := r.notice("vikunja")
if !ok || n.ToldTo != store.EndNoticeHousehold || n.ToldAt.Before(before) || n.ToldAt.After(time.Now().Add(time.Second)) {
t.Fatalf("the hub's own record does not say the household was told now: %+v (found=%v)", n, ok)
}
if len(r.to(operator)) != 0 {
t.Fatalf("the operator was mailed for a household that has an address: %+v", r.to(operator))
}
for i := 0; i < 6; i++ {
r.report(repPressed)
}
if got := r.to(household); len(got) != 1 {
t.Fatalf("one press, seven reports: %d notices, want 1", len(got))
}
if n2, _ := r.notice("vikunja"); !n2.ToldAt.Equal(n.ToldAt) {
t.Fatalf("a later report moved the told time: %s → %s", n.ToldAt, n2.ToldAt)
}
}
// 2. Two apps pressed in one minute: two notices, two records.
func TestNotice_TwoAppsInOneMinute_TwoNotices(t *testing.T) {
r := newNoticeRig(t, household)
r.report(`{"customer_id":"c1","stacks":{"deployed":["immich"]},"offsite":{"enabled":true,"ended_apps":[
{"app":"vikunja","removed_at":"2026-11-15T10:00:00Z"},{"app":"kimai","removed_at":"2026-11-15T10:00:30Z"}]}}`)
got := r.to(household)
if len(got) != 2 || !strings.Contains(got[0].subject+got[1].subject, "kimai") || !strings.Contains(got[0].subject+got[1].subject, "vikunja") {
t.Fatalf("two apps: %+v", got)
}
for _, app := range []string{"vikunja", "kimai"} {
if n, ok := r.notice(app); !ok || !n.Told() {
t.Fatalf("%s is not recorded as told: %+v", app, n)
}
}
}
// 3. A TAKE-BACK OR AN INSTALL VOIDS THE NOTICE. Press, take-back and press again within one day: the second press
// gets its own notice and its own told time — the first one's is gone. The same after an install.
func TestNotice_ATakeBackOrAnInstallVoidsTheNotice(t *testing.T) {
r := newNoticeRig(t, household)
r.report(pressed("vikunja", "2026-11-15T10:00:00Z"))
// the first notice is 40 days old in the hub's record: it would be old enough
old := time.Now().UTC().Add(-40 * 24 * time.Hour)
if err := r.st.SetEndNoticeForTest("c1", "vikunja", "2026-11-15T10:00:00Z", old, old, store.EndNoticeHousehold); err != nil {
t.Fatal(err)
}
r.report(repKept) // „keep it after all”
if n, ok := r.notice("vikunja"); ok {
t.Fatalf("after the take-back the hub still holds a notice: %+v", n)
}
r.report(pressed("vikunja", "2026-11-15T16:00:00Z")) // pressed again, the same day
if got := r.to(household); len(got) != 2 {
t.Fatalf("press, take-back, press: %d notices, want 2 (a new press needs a new notice)", len(got))
}
n, _ := r.notice("vikunja")
if !n.Told() || time.Since(n.ToldAt) > time.Minute {
t.Fatalf("the second press must have its OWN told time (now), not the first one's (40 days ago): %+v", n)
}
// an install: the app is in the app list again
if err := r.st.SetEndNoticeForTest("c1", "vikunja", "2026-11-15T16:00:00Z", old, old, store.EndNoticeHousehold); err != nil {
t.Fatal(err)
}
r.report(repInstalled)
if n, ok := r.notice("vikunja"); ok {
t.Fatalf("after an install the hub still holds a notice: %+v", n)
}
// another press that the hub never saw taken back (the tier was off in between): the press's key differs
r.report(pressed("vikunja", "2026-11-16T09:00:00Z"))
if err := r.st.SetEndNoticeForTest("c1", "vikunja", "2026-11-16T09:00:00Z", old, old, store.EndNoticeHousehold); err != nil {
t.Fatal(err)
}
r.report(`{"customer_id":"c1","stacks":{"deployed":["immich"]}}`) // the tier is off: silence is NOT a take-back
if n, ok := r.notice("vikunja"); !ok || !n.Told() {
t.Fatalf("a report that does not show the tier voided the notice: %+v", n)
}
sent := len(r.to(household))
r.report(pressed("vikunja", "2026-11-20T09:00:00Z"))
n, _ = r.notice("vikunja")
if len(r.to(household)) != sent+1 || time.Since(n.ToldAt) > time.Minute {
t.Fatalf("another press (another removed_at) must be told again and wait again: mails %d→%d, %+v", sent, len(r.to(household)), n)
}
}
// 5a. A NOTICE THAT CANNOT GO OUT tells nobody: the record stays „not told” with the reason, the next try waits an
// hour, and the first try that the mail service accepts sets the told time — then, not earlier.
func TestNotice_AFailedSendIsTriedAgainUntilItHasGone(t *testing.T) {
r := newNoticeRig(t, household)
r.fail = errors.New("resend: 503")
r.report(repPressed)
n, ok := r.notice("vikunja")
if !ok || n.Told() || !strings.Contains(n.LastReason, "did not accept") || n.Tries != 1 {
t.Fatalf("after a failed send: %+v (found=%v) — want not told, with the reason", n, ok)
}
r.fail = nil
now := time.Now()
r.h.endNotices("c1", now.Add(10*time.Minute))
if len(r.all()) != 0 {
t.Fatalf("a second try ten minutes after a failed one: %d mail(s)", len(r.all()))
}
at := now.Add(61 * time.Minute)
r.h.endNotices("c1", at)
n, _ = r.notice("vikunja")
if len(r.to(household)) != 1 || !n.Told() || n.ToldAt.Unix() != at.Unix() {
t.Fatalf("the try an hour later must send and set the told time to THAT try (%s): mails=%d %+v", at.UTC().Format(time.RFC3339), len(r.all()), n)
}
}
// 5b. The mail hold: nobody is told, the reason names the hold, and after the release the notice goes.
func TestNotice_TheMailHoldTellsNobody(t *testing.T) {
r := newNoticeRig(t, household)
dir := t.TempDir()
marker := filepath.Join(dir, mailhold.FileName)
if err := os.WriteFile(marker, nil, 0o644); err != nil {
t.Fatal(err)
}
r.d.SetMailHold(mailhold.New(dir, nil))
r.report(repPressed)
n, _ := r.notice("vikunja")
if n.Told() || !strings.Contains(n.LastReason, "MAIL-HOLD") || len(r.all()) != 0 {
t.Fatalf("under the hold: %+v, %d mail(s)", n, len(r.all()))
}
if err := os.Remove(marker); err != nil {
t.Fatal(err)
}
r.h.endNotices("c1", time.Now().Add(2*time.Hour))
if n, _ := r.notice("vikunja"); !n.Told() || len(r.to(household)) != 1 {
t.Fatalf("after the release: %+v, %d mail(s)", n, len(r.all()))
}
}
// 1b. THE DAILY LIMIT DELAYS A NOTICE; IT NEVER DROPS ONE. Seven apps in one report: five are told today, two stay
// „not told” with the reason — and are told a day later.
func TestNotice_TheDailyLimitDelaysAndNeverDrops(t *testing.T) {
r := newNoticeRig(t, household)
var recs []string
for i := 0; i < 7; i++ {
recs = append(recs, fmt.Sprintf(`{"app":"app-%d","removed_at":"2026-11-15T10:00:00Z"}`, i))
}
r.report(`{"customer_id":"c1","stacks":{"deployed":["immich"]},"offsite":{"enabled":true,"ended_apps":[` + strings.Join(recs, ",") + `]}}`)
if got := r.to(household); len(got) != 5 {
t.Fatalf("seven presses: %d notices today, want the daily limit 5", len(got))
}
untold := 0
for i := 0; i < 7; i++ {
if n, ok := r.notice(fmt.Sprintf("app-%d", i)); !ok {
t.Fatalf("app-%d has no record — a notice was DROPPED", i)
} else if !n.Told() {
untold++
if !strings.Contains(n.LastReason, "daily limit") {
t.Fatalf("app-%d: not told, and the reason is %q", i, n.LastReason)
}
}
}
if untold != 2 {
t.Fatalf("%d of 7 are not told, want 2", untold)
}
if err := r.st.BackdateNotificationLogForTest("c1", 25*time.Hour); err != nil {
t.Fatal(err)
}
r.h.endNotices("c1", time.Now().Add(25*time.Hour))
if got := r.to(household); len(got) != 7 {
t.Fatalf("a day later: %d notices in all, want 7 — the limit must only delay", len(got))
}
for i := 0; i < 7; i++ {
if n, _ := r.notice(fmt.Sprintf("app-%d", i)); !n.Told() {
t.Fatalf("app-%d is still not told", i)
}
}
}
// 3 (A.3). A HOUSEHOLD WITH NO ADDRESS: the operator is told once per app, operator-only, and THAT mail starts the
// 30 days. The address is added after the press: the household gets its own notice and the 30 days start again.
func TestNotice_NoAddress_TheOperatorIsToldInItsPlace_AndAnAddressLaterRestarts(t *testing.T) {
r := newNoticeRig(t, "")
r.report(repPressed)
op := r.opNotice(offsitekeys.EventEndNoticeOperator)
if len(op) != 1 || !strings.Contains(op[0].body, `"vikunja"`) || !strings.Contains(op[0].body, "c1") || !strings.Contains(op[0].body, "no notification address") {
t.Fatalf("the operator must be told once, naming the household, the app and why: %+v", r.all())
}
n, _ := r.notice("vikunja")
if n.ToldTo != store.EndNoticeOperator || !n.Told() {
t.Fatalf("the record must say the OPERATOR was told: %+v", n)
}
if e := r.events(offsitekeys.EventEndNoticeOperator); len(e) != 1 || e[0].Severity != "warning" || !strings.Contains(e[0].DetailsJSON, `"subject":"vikunja"`) {
t.Fatalf("the operator's event row: %+v", e)
}
for i := 0; i < 5; i++ {
r.report(repPressed)
}
if got := r.opNotice(offsitekeys.EventEndNoticeOperator); len(got) != 1 {
t.Fatalf("told once per app: %d mails after six reports", len(got))
}
// the operator's notice is 40 days old; the household gets an address now
old := time.Now().UTC().Add(-40 * 24 * time.Hour)
if err := r.st.SetEndNoticeForTest("c1", "vikunja", "2026-11-15T10:00:00Z", old, old, store.EndNoticeOperator); err != nil {
t.Fatal(err)
}
if err := r.st.SaveNotificationPrefs("c1", household, []string{}, 6); err != nil {
t.Fatal(err)
}
r.report(repPressed)
n, _ = r.notice("vikunja")
if len(r.to(household)) != 1 || n.ToldTo != store.EndNoticeHousehold || time.Since(n.ToldAt) > time.Minute {
t.Fatalf("an address added after the press: the household must be told and the 30 days start again: mails=%+v record=%+v", r.mails, n)
}
// a blocked household is the same case
r2 := newNoticeRig(t, household)
if err := r2.st.SetCustomerConfigStatus("c1", "blocked"); err != nil {
t.Fatal(err)
}
r2.report(repPressed)
if n, _ := r2.notice("vikunja"); n.ToldTo != store.EndNoticeOperator || len(r2.to(household)) != 0 || len(r2.opNotice(offsitekeys.EventEndNoticeOperator)) != 1 {
t.Fatalf("a blocked household: record %+v, mails %+v", n, r2.all())
}
}
// 3b. The operator cannot be told either: NOBODY was told, so nothing starts.
func TestNotice_NoAddress_AndTheOperatorsMailFails_NobodyWasTold(t *testing.T) {
r := newNoticeRig(t, "")
r.fail = errors.New("resend: 503")
r.report(repPressed)
n, ok := r.notice("vikunja")
if !ok || n.Told() || !strings.Contains(n.LastReason, "operator could not be told") {
t.Fatalf("no address and the operator's mail failed: %+v", n)
}
if e := r.events(offsitekeys.EventEndNoticeOperator); len(e) != 0 {
t.Fatalf("an event row says the operator was told, and no mail went: %+v", e)
}
}
// 5c. A notice that has told nobody for three days: the operator gets ONE line. Not at two days; not twice; not
// while the hold is on.
func TestNotice_StuckForThreeDays_TheOperatorGetsOneLine(t *testing.T) {
r := newNoticeRig(t, household)
r.fail = errors.New("resend: 503")
r.report(repPressed)
now := time.Now()
stuck := func() []store.Event { return r.events(offsitekeys.EventEndNoticeStuck) }
r.h.endNotices("c1", now.Add(2*24*time.Hour))
if e := stuck(); len(e) != 0 {
t.Fatalf("told the operator after two days: %+v", e)
}
dir := t.TempDir()
marker := filepath.Join(dir, mailhold.FileName)
if err := os.WriteFile(marker, nil, 0o644); err != nil {
t.Fatal(err)
}
r.d.SetMailHold(mailhold.New(dir, nil))
r.h.endNotices("c1", now.Add(EndNoticeStuckAfter+time.Hour))
if e := stuck(); len(e) != 0 {
t.Fatalf("the line was filed while the hold is on: %+v", e)
}
_ = os.Remove(marker)
r.h.endNotices("c1", now.Add(EndNoticeStuckAfter+3*time.Hour))
e := stuck()
if len(e) != 1 || e[0].Severity != "warning" || !strings.Contains(e[0].Message, `"vikunja"`) || !strings.Contains(e[0].Message, "did not accept") || !strings.Contains(e[0].DetailsJSON, `"subject":"vikunja"`) {
t.Fatalf("want ONE warning naming the app and the reason: %+v", e)
}
for i := 5; i < 10; i++ {
r.h.endNotices("c1", now.Add(EndNoticeStuckAfter+time.Duration(i)*time.Hour))
}
if e := stuck(); len(e) != 1 {
t.Fatalf("the line was repeated: %d", len(e))
}
if n, _ := r.notice("vikunja"); n.Told() {
t.Fatalf("the operator's line must not count as the notice: %+v", n)
}
}
// 7. THE DATABASE IS PUT BACK FROM A COPY THAT LACKS THE NOTICE: the hub has no record, so it tells again and the
// told time is new — it waits again.
func TestNotice_ADatabaseWithoutTheNoticeTellsAgain(t *testing.T) {
r := newNoticeRig(t, household)
r.report(repPressed)
if len(r.to(household)) != 1 {
t.Fatal("fixture: no first notice")
}
restored := newNoticeRig(t, household) // the copy: the household and its address, no notice row
if _, ok := restored.notice("vikunja"); ok {
t.Fatal("fixture: the copy holds a notice")
}
restored.report(repPressed)
n, ok := restored.notice("vikunja")
if !ok || !n.Told() || time.Since(n.ToldAt) > time.Minute || len(restored.to(household)) != 1 {
t.Fatalf("on the copy the household must be told again, with a told time of now: %+v mails=%d", n, len(restored.all()))
}
}
// 7. A household deleted between the notice and the window: its notice rows go with it, and nothing is mailed after.
func TestNotice_ADeletedHouseholdHasNoNotice(t *testing.T) {
r := newNoticeRig(t, household)
r.fail = errors.New("resend: 503")
r.report(repPressed)
if err := r.st.PurgeCustomerResidue("c1"); err != nil {
t.Fatal(err)
}
if ns, _ := r.st.EndNotices("c1"); len(ns) != 0 {
t.Fatalf("the notice outlived the household: %+v", ns)
}
r.fail = nil
r.h.endNotices("c1", time.Now().Add(2*time.Hour))
if len(r.all()) != 0 {
t.Fatalf("a deleted household was mailed: %+v", r.all())
}
}
// A box's names are the box's word: a name that is not an app name, a reserved name and an installed app get no
// notice; and no box can POST the hub's own two warnings.
func TestNotice_OnlyRealRemovedAppsAreTold(t *testing.T) {
r := newNoticeRig(t, household)
r.report(`{"customer_id":"c1","stacks":{"deployed":["immich"]},"offsite":{"enabled":true,"ended_apps":[
{"app":"<b>x</b>"},{"app":"_shares"},{"app":"traefik"},{"app":"immich"},{"app":"pay now at evil.example"}]}}`)
if len(r.all()) != 0 {
t.Fatalf("mails: %+v", r.all())
}
if ns, _ := r.st.EndNotices("c1"); len(ns) != 0 {
t.Fatalf("rows: %+v", ns)
}
for _, typ := range []string{offsitekeys.EventEndNoticeOperator, offsitekeys.EventEndNoticeStuck} {
if allowedEventTypes[typ] {
t.Fatalf("a box must not be able to POST %s", typ)
}
}
}
// THE SEAM IS WIRED: with the real spawn (off the request) a report still causes the notice.
func TestNotice_TheReportStartsTheWalkOffTheRequest(t *testing.T) {
r := newNoticeRig(t, household)
endNoticeSpawn = func(f func()) { go f() } // newNoticeRig's cleanup restores the original
r.report(repPressed)
deadline := time.Now().Add(3 * time.Second)
for time.Now().Before(deadline) {
if len(r.to(household)) == 1 {
if n, _ := r.notice("vikunja"); n.Told() {
return
}
}
time.Sleep(10 * time.Millisecond)
}
t.Fatalf("no notice within 3 s of the report: %+v", r.all())
}
+11 -167
View File
@@ -32,9 +32,6 @@ type Dispatcher struct {
mu sync.Mutex
opCooldowns map[string]time.Time // "customerID:eventType" → last operator notify
custCooldowns map[string]time.Time // "customerID:eventType" → last customer notify
// receiptDropDay: per household, the UTC day on which a dropped receipt was last said in the log (once a day).
receiptDropDay map[string]string
// sendEmailFn is the email sender, seam-injected so tests exercise routing without real HTTP.
// Defaults to (*Dispatcher).sendEmail (Resend) in NewDispatcher. headers (nil = none) become
// Resend custom headers — used for the high-priority nudge on error/critical mails (v0.71.0,
@@ -246,11 +243,11 @@ func (d *Dispatcher) processEvent(customerID, eventType, severity, message, mess
return
}
// D84 (`09` §3 decision 309): a household RECEIPT — the mail that tells the household what it has just pressed.
// Before the severity gate (the event is `info`: nothing is wrong) and with no operator mail: the event row is
// the operator's record.
// D84 + the notice lock (`09` §3 decisions 309 and 315): the household's RECEIPT is NOT sent from the box's
// event. The hub sends it itself, from its own record of the box's reports (api.endNotices → SendEndNotice),
// until it has gone — a lost, repeated or invented event changes nothing. The event row is the operator's record.
if householdReceiptEvents[eventType] {
d.processReceipt(customerID, eventType, severity, detailsJSON)
d.logger.Printf("[DEBUG] %s from the box of %s is recorded, not mailed — the hub sends this notice itself from the box's report", eventType, customerID)
return
}
@@ -649,6 +646,8 @@ var perSubjectCooldownEvents = map[string]bool{
// one never-seen app is one mail (subject = the app); one window's large request is one mail (subject = the window)
"offsite_end_claim_unseen": true,
"offsite_ended_large_agreement": true,
// the notice lock: one line per app whose notice cannot go out (subject = the app)
"offsite_end_notice_stuck": true,
}
// cooldownSubjectSuffix returns ":"+subject for an allow-listed type whose details carry a non-empty `subject`.
@@ -775,166 +774,6 @@ var recordOnlyEvents = map[string]bool{
"recovery_unit_capture_failed": true,
}
// householdReceiptEvents (D84, `09` §3 decision 309) are mails that confirm the household's OWN press of a button
// whose effect comes later and cannot be undone after it. One type today: `offsite_end_scheduled` — the household
// pressed „delete backup data” (or „delete in 30 days”), and the mail names the app, the day the off-site copies are
// deleted and how to take it back. The reviewer's reason, accepted by the operator: the page alone does not warn a
// household whose login someone else uses.
//
// A receipt bypasses the household's group switches (whoever holds the login could switch a group off first) — so it
// is the one household mail a BOX can cause whatever the household chose, and it is built so a box can do nothing
// else with it (review of 2026-11-15; processReceipt):
// - THE HUB WRITES IT. Subject and body come only from the hub's own bundle, with an app name the hub checked
// (the shape of an app name, not reserved) and dates the hub parsed. The box's `message` is never mailed.
// - ONE PER PRESS. The cooldown key is the checked app + the parsed time of the press — not the box's free
// `subject` — so one press delivered twice is one mail, and two apps removed in one hour are two.
// - AT MOST receiptsPerDay PER HOUSEHOLD IN 24 HOURS, counted in the hub's own sent log (it survives a restart).
// The rest are dropped; the hub says so once a day per household.
// - to the household's own notification address by the customer leg's own delivery; never to the operator.
//
// A household with NO address gets no mail — the page is all it has (`07` §8.2f). A blocked household gets none.
// Pinned by TestD84_Receipt*.
var householdReceiptEvents = map[string]bool{
"offsite_end_scheduled": true,
}
// receiptsPerDay is the most receipts ONE household is mailed in 24 hours (review of 2026-11-15).
const receiptsPerDay = 5
// receiptCooldown: the same press is never mailed twice within this time.
const receiptCooldown = 24 * time.Hour
// receipt is what the hub accepts of a receipt's details.
type receipt struct {
App string
Removed time.Time
Ends time.Time
}
// parseReceipt checks a receipt's details. ok=false → nothing is mailed. The app must have the shape of an app name
// and not be reserved (store.ValidPresenceApp / ReservedPresenceApp — the same check the hub's app record uses); the
// two dates must be times, and the end must be after the press and at most 40 days after it.
func parseReceipt(detailsJSON string) (receipt, bool) {
var d struct {
StackName string `json:"stack_name"`
RemovedAt string `json:"removed_at"`
EndsAt string `json:"ends_at"`
}
if detailsJSON == "" || json.Unmarshal([]byte(detailsJSON), &d) != nil {
return receipt{}, false
}
if !store.ValidPresenceApp(d.StackName) || store.ReservedPresenceApp(d.StackName) {
return receipt{}, false
}
removed, err1 := time.Parse(time.RFC3339, d.RemovedAt)
ends, err2 := time.Parse(time.RFC3339, d.EndsAt)
if err1 != nil || err2 != nil || !ends.After(removed) || ends.Sub(removed) > 40*24*time.Hour {
return receipt{}, false
}
return receipt{App: d.StackName, Removed: removed.UTC(), Ends: ends.UTC()}, true
}
// receiptDate writes a day in the household's date form (the controller's i18n.FormatDate), in Budapest time.
func receiptDate(lang string, t time.Time) string {
if budapest != nil {
t = t.In(budapest)
}
if i18n.Normalize(lang) == "en" {
return t.Format("2006-01-02")
}
return t.Format("2006. 01. 02.")
}
// FormatReceiptEmail is the receipt's whole text — the hub's own words and the checked details, nothing of the box's.
func FormatReceiptEmail(lang, customerID, eventType, severity string, rc receipt) (string, string) {
b := i18n.Shared()
lang = i18n.Normalize(lang)
sentence := b.Msgf(lang, "mail.receipt."+eventType, rc.App, receiptDate(lang, rc.Ends))
// The details the frame reads (the app for the headline) are rebuilt from the CHECKED values.
det, _ := json.Marshal(map[string]string{"stack_name": rc.App})
return FormatCustomerEmail(lang, customerID, eventType, severity, sentence, "", string(det))
}
// processReceipt mails ONE receipt to the household, or drops it — see householdReceiptEvents.
func (d *Dispatcher) processReceipt(customerID, eventType, severity, detailsJSON string) {
rc, ok := parseReceipt(detailsJSON)
if !ok {
d.receiptDropped(customerID, eventType, "its details do not name an app and two dates the hub accepts")
return
}
if d.store.IsCustomerBlocked(customerID) {
return
}
prefs, err := d.store.GetNotificationPrefs(customerID)
if err != nil || prefs == nil || prefs.Email == "" {
return // no address: the page is all the household has
}
key := customerID + ":" + eventType + ":" + rc.App + "|" + rc.Removed.Format(time.RFC3339)
now := time.Now()
sentToday, cerr := d.store.CountCustomerSentSince(customerID, eventType, now.Add(-24*time.Hour))
if cerr != nil {
sentToday = receiptsPerDay // a log that cannot be read is not „nothing sent”: no receipt goes
}
d.mu.Lock()
// The map holds only presses of the last day: older keys of this household's receipts go now.
prefix, inFlight := customerID+":"+eventType+":", 0
for k, at := range d.custCooldowns {
if strings.HasPrefix(k, prefix) {
if now.Sub(at) >= receiptCooldown {
delete(d.custCooldowns, k)
} else {
inFlight++
}
}
}
if _, dup := d.custCooldowns[key]; dup {
d.mu.Unlock()
d.logger.Printf("[INFO] Receipt %s for %s/%s skipped — this press was mailed already", eventType, customerID, rc.App)
return
}
// The limit counts what the log says was sent AND what is reserved in memory (a burst is processed faster than
// the log is written).
if sentToday >= receiptsPerDay || inFlight >= receiptsPerDay {
d.mu.Unlock()
d.receiptDropped(customerID, eventType, fmt.Sprintf("the household already got %d receipt(s) in the last 24 hours (the limit)", receiptsPerDay))
return
}
d.custCooldowns[key] = now
d.mu.Unlock()
subject, body := FormatReceiptEmail(d.store.CustomerLanguage(customerID), customerID, eventType, severity, rc)
logMsg := eventType + " " + rc.App
if err := d.deliver("customer receipt "+eventType, customerID, prefs.Email, subject, body, priorityHeaders(severity)); isHeld(err) {
d.forgetCustomerCooldown(key)
d.store.LogNotification(customerID, eventType, severity, logMsg, "held", "MAIL-HOLD marker present — dropped", "customer")
return
} else if err != nil {
d.logger.Printf("[ERROR] Receipt %s failed for %s: %v", eventType, customerID, err)
d.store.LogNotification(customerID, eventType, severity, logMsg, "failed", err.Error(), "customer")
return
}
d.logger.Printf("[INFO] Receipt %s sent for %s/%s", eventType, customerID, rc.App) // never the address (R-922)
d.store.LogNotification(customerID, eventType, severity, logMsg, "sent", "", "customer")
}
// receiptDropped says ONCE per household and day that receipts are being dropped, and why — in the hub's log and as
// one `skipped` row. A box that posts a thousand of them gets one line, not a thousand.
func (d *Dispatcher) receiptDropped(customerID, eventType, why string) {
day := time.Now().UTC().Format("2006-01-02")
d.mu.Lock()
if d.receiptDropDay == nil {
d.receiptDropDay = map[string]string{}
}
already := d.receiptDropDay[customerID] == day
d.receiptDropDay[customerID] = day
d.mu.Unlock()
if already {
return
}
d.logger.Printf("[WARN] Receipt %s from the box of %s was NOT mailed: %s. Further drops for this household are not logged today.", eventType, customerID, why)
d.store.LogNotification(customerID, eventType, "info", eventType, "skipped", "receipt dropped: "+why, "customer")
}
// operatorOnlyEvents are event types that must NEVER reach a customer, whatever their preferences say.
//
// R-97c. This register exists because the guarantee it provides was previously ASSERTED IN A COMMENT
@@ -1044,6 +883,11 @@ var operatorOnlyEvents = map[string]bool{
// before the hub could agree), and a window was asked for more than half of the store. Hub-origin only.
"offsite_end_claim_unseen": true,
"offsite_ended_large_agreement": true,
// The notice lock (`09` §3 decision 315): the operator is told IN PLACE of a household that has no usable
// address (that mail starts the app's 30 days), and told once when a notice could not go out for days.
// Hub-origin only: NOT in api.allowedEventTypes.
"offsite_end_notice_operator": true,
"offsite_end_notice_stuck": true,
// OS updates (hub v0.130.0, `11` §8 step 2): run failures, rings, switches and approvals are operator facts.
// os_update_applied is deliberately NOT here — it is the household's one line (info: recorded, never mailed).
"os_update_failed": true,
+162
View File
@@ -0,0 +1,162 @@
package notify
import (
"encoding/json"
"fmt"
"strings"
"time"
"gitea.dooplex.hu/admin/felhom-hub/internal/i18n"
"gitea.dooplex.hu/admin/felhom-hub/internal/store"
)
// householdReceiptEvents (D84, `09` §3 decision 309) are mails that tell the household about its OWN press of a button
// whose effect comes later and cannot be undone after it. One type today: `offsite_end_scheduled` — the household
// pressed „delete backup data” (or „delete in 30 days”), and the mail names the app, the day from which the off-site
// copies are deleted, how to take it back, and what to do if somebody else pressed it. The reviewer's reason,
// accepted by the operator: the page alone does not warn a household whose login someone else uses.
//
// THE NOTICE LOCK (`09` §3 decision 315; `07-backup-architecture.md` §8.2f): the mail is a CONDITION of the deletion,
// so it is not sent from the box's event (an event is lost when the hub is away for nine seconds — R-1361 — and a
// failed send was never tried again). The hub sends it from its own state: api.endNotices calls SendEndNotice for
// every app its record (store.offsite_end_notices) says is named in a report and not told, at every report, until the
// mail service accepts it; only then does the record say „told”, and only 30 days after that does the hub agree.
//
// A receipt bypasses the household's group switches (whoever holds the login could switch a group off first), so it
// is built so that a box can do nothing else with it:
// - THE HUB WRITES IT. Subject and body come only from the hub's own bundle, with an app name the hub checked
// (the shape of an app name, not reserved) and a day the hub computed from its own clock. Nothing of the box's.
// - AT MOST receiptsPerDay PER HOUSEHOLD IN 24 HOURS, counted in the hub's own sent log (it survives a restart).
// The limit DELAYS a notice — the record stays „not told” and the next report tries again — it never drops one.
// - to the household's own notification address; never to the operator.
//
// A household with no address, or a blocked one, gets no mail: the caller then tells the operator in its place
// (`offsite_end_notice_operator`), and sends the household's mail when an address appears. Pinned by TestNotice_*.
var householdReceiptEvents = map[string]bool{
"offsite_end_scheduled": true,
}
// EndNoticeEvent is the type the household's notice is logged and counted under.
const EndNoticeEvent = "offsite_end_scheduled"
// receiptsPerDay is the most receipts ONE household is mailed in 24 hours (review of 2026-11-15).
const receiptsPerDay = 5
// EndNoticeOutcome is what one try of the household's notice came to. Only EndNoticeSent tells the household.
type EndNoticeOutcome string
const (
EndNoticeSent EndNoticeOutcome = "sent"
EndNoticeNoAddress EndNoticeOutcome = "no-address" // no notification address, or the household is blocked
EndNoticeLimit EndNoticeOutcome = "limit" // the household's daily limit: later
EndNoticeHeld EndNoticeOutcome = "held" // the MAIL-HOLD marker
EndNoticeFailed EndNoticeOutcome = "failed" // the mail service did not accept it, or is not configured
)
// receiptDate writes a day in the household's date form (the controller's i18n.FormatDate), in Budapest time.
func receiptDate(lang string, t time.Time) string {
if budapest != nil {
t = t.In(budapest)
}
if i18n.Normalize(lang) == "en" {
return t.Format("2006-01-02")
}
return t.Format("2006. 01. 02.")
}
// FormatReceiptEmail is the receipt's whole text — the hub's own words, a checked app name and the hub's own day.
// The household mail frame with the hub's sentence as the message line (FormatCustomerEmail).
func FormatReceiptEmail(lang, customerID, eventType, severity, app string, ends time.Time) (string, string) {
b := i18n.Shared()
lang = i18n.Normalize(lang)
day := receiptDate(lang, ends)
sentence := b.Msgf(lang, "mail.receipt."+eventType, app, day)
// The details the frame reads (the app for the headline) are rebuilt from the CHECKED value.
det, _ := json.Marshal(map[string]string{"stack_name": app})
return FormatCustomerEmail(lang, customerID, eventType, severity, sentence, "", string(det))
}
// MailHeld reports whether the hub-wide mail hold is on (no mail of any kind goes out).
func (d *Dispatcher) MailHeld() bool { return d.mailHold.Held() }
// SendEndNotice tries ONCE to mail the household that the off-site copies of `app` are deleted from the day `ends`
// (the hub's own day: the caller passes its clock + 30 days). It returns what the try came to and, when nobody was
// told, the hub's own words for why. It keeps no state: the caller's record decides whether there is a next try.
// app must have passed store.ValidPresenceApp; a name that has not is refused here again.
func (d *Dispatcher) SendEndNotice(customerID, app string, ends time.Time) (EndNoticeOutcome, string) {
if !store.ValidPresenceApp(app) || store.ReservedPresenceApp(app) {
return EndNoticeFailed, "not an app name"
}
if d.resendAPIKey == "" {
return EndNoticeFailed, "the hub has no mail service configured"
}
if d.store.IsCustomerBlocked(customerID) {
return EndNoticeNoAddress, "the household is blocked"
}
prefs, err := d.store.GetNotificationPrefs(customerID)
if err != nil {
return EndNoticeFailed, "the household's notification address could not be read"
}
if prefs == nil || strings.TrimSpace(prefs.Email) == "" {
return EndNoticeNoAddress, "the household has no notification address"
}
now := time.Now()
sentToday, cerr := d.store.CountCustomerSentSince(customerID, EndNoticeEvent, now.Add(-24*time.Hour))
if cerr != nil {
// A log that cannot be read is not „nothing sent”: no receipt goes, and the notice stays to be tried again.
return EndNoticeLimit, "the hub's sent log could not be read"
}
if sentToday >= receiptsPerDay {
return EndNoticeLimit, fmt.Sprintf("the household already got %d of these notices in the last 24 hours (the daily limit) — this one goes later", receiptsPerDay)
}
subject, body := FormatReceiptEmail(d.store.CustomerLanguage(customerID), customerID, EndNoticeEvent, "info", app, ends)
logMsg := EndNoticeEvent + " " + app
if err := d.deliver("customer receipt "+EndNoticeEvent, customerID, strings.TrimSpace(prefs.Email), subject, body, nil); isHeld(err) {
d.store.LogNotification(customerID, EndNoticeEvent, "info", logMsg, "held", "MAIL-HOLD marker present — not sent; the notice stays to be sent", "customer")
return EndNoticeHeld, "the hub is not sending mail (the MAIL-HOLD marker is present)"
} else if err != nil {
d.logger.Printf("[ERROR] Notice %s failed for %s/%s: %v — it stays to be sent", EndNoticeEvent, customerID, app, err)
d.store.LogNotification(customerID, EndNoticeEvent, "info", logMsg, "failed", err.Error(), "customer")
return EndNoticeFailed, "the mail service did not accept the mail"
}
d.logger.Printf("[INFO] Notice %s sent for %s/%s", EndNoticeEvent, customerID, app) // never the address (R-922)
if lerr := d.store.LogNotification(customerID, EndNoticeEvent, "info", logMsg, "sent", "", "customer"); lerr != nil {
d.logger.Printf("[WARN] Notice %s for %s/%s was sent, but the sent log could not be written: %v", EndNoticeEvent, customerID, app, lerr)
}
return EndNoticeSent, ""
}
// SendOperatorNow mails the operator ONE hub-made event and returns only when the mail service has answered: nil
// means it accepted the mail. For an operator mail whose DELIVERY is a condition of something else (the notice that
// starts an app's 30 days when the household has no address) — the ordinary operator path (processOperator) answers
// at once and retries in the background, so its caller cannot know. No cooldown: the caller's own record tells each
// thing once. The event type must be operator-only.
func (d *Dispatcher) SendOperatorNow(customerID, eventType, severity, message, detailsJSON string) error {
if !operatorOnlyEvents[eventType] {
return fmt.Errorf("notify: %s is not an operator-only type", eventType)
}
if d.resendAPIKey == "" {
return fmt.Errorf("the hub has no mail service configured")
}
if !d.operatorOn || d.operatorEmail == "" {
return fmt.Errorf("the operator's mail is switched off")
}
subject, body := FormatOperatorEmail(customerID, eventType, severity, message, detailsJSON)
if err := d.deliver("operator mail "+eventType, "operator", d.operatorEmail, subject, body, priorityHeaders(severity)); isHeld(err) {
d.store.LogNotification(customerID, eventType, severity, message, "held", "MAIL-HOLD marker present — not sent", "operator")
return err
} else if err != nil {
d.logger.Printf("[ERROR] Operator email failed for %s/%s: %v", customerID, eventType, err)
d.store.LogNotification(customerID, eventType, severity, message, "failed", err.Error(), "operator")
return err
}
d.logger.Printf("[INFO] Operator email sent for %s/%s", customerID, eventType)
d.store.LogNotification(customerID, eventType, severity, message, "sent", "", "operator")
return nil
}
// SetSendFnForTest replaces the function that hands a mail to the mail service. TEST-ONLY: it lets a test in another
// package (api.endNotices) see every mail and fail a send, with no call to the real service.
func (d *Dispatcher) SetSendFnForTest(fn func(to, subject, body string, headers map[string]string) error) {
d.sendEmailFn = fn
}
+262
View File
@@ -0,0 +1,262 @@
package notify
import (
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"time"
"gitea.dooplex.hu/admin/felhom-hub/internal/mailhold"
)
// THE NOTICE LOCK, the mail's half (`09` §3 decisions 315 and 316; `07-backup-architecture.md` §8.2f). The household's
// notice is sent by the hub from its own state (SendEndNotice, called by api.endNotices) — never from the box's event.
// These tests replace TestD84_Receipt_* of the first build, whose mail left from the event.
// RED-PROOFS: documentation/audits/build-release-4b-2026-11-15/A/redproof-notice-mail.txt.
type receiptMail struct{ to, subject, body string }
func receiptDispatcher(t *testing.T, prefsEmail string, enabled []string) (*Dispatcher, *[]receiptMail) {
t.Helper()
st := opOnlyStore(t)
if prefsEmail != "" || enabled != nil {
if err := st.SaveNotificationPrefs("c1", prefsEmail, enabled, 6); err != nil {
t.Fatal(err)
}
}
d := opOnlyDispatcher(t, st, &sentTo{})
var mails []receiptMail
d.sendEmailFn = func(to, subject, body string, _ map[string]string) error {
mails = append(mails, receiptMail{to, subject, body})
return nil
}
return d, &mails
}
// noticeDay: 2026-12-15 12:00 UTC — the same calendar day in Budapest.
var noticeDay = time.Date(2026, 12, 15, 12, 0, 0, 0, time.UTC)
const receiptDetails = `{"stack_name":"vikunja","removed_at":"2026-11-15T10:00:00Z","ends_at":"2026-12-15T10:00:00Z","subject":"vikunja|2026-11-15T10:00:00Z"}`
// The notice reaches the household's own address although EVERY group switch is off. Its subject names the deletion
// and its day (not „Information”); its body names the day, the way back, and what to do if the household did not ask
// for this. The operator is not mailed, and no raw details are in it.
func TestNotice_ReachesTheHouseholdWhateverItsSwitches(t *testing.T) {
d, mails := receiptDispatcher(t, "customer@example.com", []string{})
if out, why := d.SendEndNotice("c1", "vikunja", noticeDay); out != EndNoticeSent {
t.Fatalf("outcome = %s (%s), want sent", out, why)
}
if len(*mails) != 1 {
t.Fatalf("mails = %d, want exactly one (to the household): %+v", len(*mails), *mails)
}
m := (*mails)[0]
if m.to != "customer@example.com" {
t.Fatalf("the notice went to %q, not to the household's own address", m.to)
}
if !strings.Contains(m.subject, "vikunja: a távoli másolatok törlésre kijelölve") {
t.Fatalf("the subject does not name the app: %q", m.subject)
}
for _, want := range []string{"Törlésük napja: 2026. 12. 15. Addig", "Mégis megtartom", "Mentések oldalon"} {
if !strings.Contains(m.body, want) {
t.Fatalf("the body lacks %q:\n%s", want, m.body)
}
}
if strings.Contains(m.body, "removed_at") || strings.Contains(m.body, "{") {
t.Fatalf("the body carries raw details:\n%s", m.body)
}
// the hub's own sent log holds it (the daily limit counts there)
if n, err := d.store.CountCustomerSentSince("c1", EndNoticeEvent, time.Now().Add(-time.Hour)); err != nil || n != 1 {
t.Fatalf("sent log rows = %d (%v), want 1", n, err)
}
}
// An English household reads it in English, with the English date form.
func TestNotice_FollowsTheHouseholdsLanguage(t *testing.T) {
d, mails := receiptDispatcher(t, "customer@example.com", nil)
if err := d.store.SaveReport("c1", []byte(`{"customer_id":"c1","language":"en"}`)); err != nil {
t.Fatal(err)
}
if out, _ := d.SendEndNotice("c1", "vikunja", noticeDay); out != EndNoticeSent || len(*mails) != 1 {
t.Fatalf("outcome %s, mails %d", out, len(*mails))
}
m := (*mails)[0]
if !strings.Contains(m.subject, "vikunja: the off-site copies are marked for deletion") {
t.Fatalf("not the English subject: %q", m.subject)
}
for _, want := range []string{"They are deleted on: 2026-12-15.", "Keep it after all"} {
if !strings.Contains(m.body, want) {
t.Fatalf("the English body lacks %q:\n%s", want, m.body)
}
}
if strings.Contains(m.body, "Mégis megtartom") {
t.Fatalf("Hungarian in an English household's mail:\n%s", m.body)
}
}
// THE BOX'S EVENT MAILS NOBODY — whether it arrives once, twice, with invented words or never. (The wrong cases „the
// event never arrives” and „it arrives twice”: the notice does not depend on it.)
func TestNotice_TheBoxesEventMailsNobody(t *testing.T) {
d, mails := receiptDispatcher(t, "customer@example.com", []string{})
evil := "ANY WORDS THE BOX CHOOSES: pay at http://evil.example"
d.ProcessBoxEvent("c1", "offsite_end_scheduled", "info", "x", "", receiptDetails, "controller")
d.ProcessBoxEvent("c1", "offsite_end_scheduled", "info", "x", "", receiptDetails, "controller")
for i := 0; i < 50; i++ {
d.ProcessBoxEvent("c1", "offsite_end_scheduled", "warning", evil, evil,
fmt.Sprintf(`{"stack_name":"app-%d","removed_at":"2026-11-15T10:00:00Z","ends_at":"2026-12-15T10:00:00Z"}`, i), "controller")
}
if len(*mails) != 0 {
t.Fatalf("the box's event caused %d mail(s); first: %+v", len(*mails), (*mails)[0])
}
// positive control: the same dispatcher does mail an ordinary event of the box
if err := d.store.SaveNotificationPrefs("c1", "customer@example.com", []string{"app_stopped_unhealthy"}, 6); err != nil {
t.Fatal(err)
}
d.ProcessBoxEvent("c1", "app_stopped_unhealthy", "error", "x", "", `{"stack_name":"vikunja"}`, "controller")
if len(*mails) == 0 {
t.Fatal("control: the dispatcher mails nothing at all — the test above proves nothing")
}
}
// A household with no address, and a blocked one: nobody is mailed here, and the outcome says „no address” so that
// the caller tells the operator in its place.
func TestNotice_NoAddressOrBlocked_SaysSoAndMailsNobody(t *testing.T) {
d, mails := receiptDispatcher(t, "", nil)
if out, _ := d.SendEndNotice("c1", "vikunja", noticeDay); out != EndNoticeNoAddress || len(*mails) != 0 {
t.Fatalf("no address: outcome %s, %d mail(s)", out, len(*mails))
}
d, mails = receiptDispatcher(t, "customer@example.com", nil)
if err := d.store.SetCustomerConfigStatus("c1", "blocked"); err != nil {
t.Fatal(err)
}
if out, _ := d.SendEndNotice("c1", "vikunja", noticeDay); out != EndNoticeNoAddress || len(*mails) != 0 {
t.Fatalf("blocked: outcome %s, %d mail(s)", out, len(*mails))
}
if err := d.store.SetCustomerConfigStatus("c1", "active"); err != nil {
t.Fatal(err)
}
if out, _ := d.SendEndNotice("c1", "vikunja", noticeDay); out != EndNoticeSent || len(*mails) != 1 {
t.Fatalf("positive control: an unblocked household must get the notice: outcome %s, %d mail(s)", out, len(*mails))
}
}
// The daily limit DELAYS: the sixth notice of 24 hours is not sent and says „limit”; a hub restart (a new dispatcher
// on the same store) does not give five more; a day later it goes.
func TestNotice_TheDailyLimitDelaysAndSurvivesARestart(t *testing.T) {
d, mails := receiptDispatcher(t, "customer@example.com", nil)
for i := 0; i < receiptsPerDay; i++ {
if out, why := d.SendEndNotice("c1", fmt.Sprintf("app-%d", i), noticeDay); out != EndNoticeSent {
t.Fatalf("notice %d: %s (%s)", i, out, why)
}
}
if out, why := d.SendEndNotice("c1", "one-more", noticeDay); out != EndNoticeLimit || !strings.Contains(why, "later") {
t.Fatalf("the sixth notice: outcome %s (%q), want limit", out, why)
}
d2 := opOnlyDispatcher(t, d.store, &sentTo{}) // the restart: nothing in memory
n2 := 0
d2.sendEmailFn = func(string, string, string, map[string]string) error { n2++; return nil }
if out, _ := d2.SendEndNotice("c1", "one-more", noticeDay); out != EndNoticeLimit || n2 != 0 {
t.Fatalf("after a restart: outcome %s, %d mail(s) — the limit must be the hub's sent log", out, n2)
}
if len(*mails) != receiptsPerDay {
t.Fatalf("mails = %d, want %d", len(*mails), receiptsPerDay)
}
// a day later the log's rows are older than 24 hours: the delayed notice goes
if err := d.store.BackdateNotificationLogForTest("c1", 25*time.Hour); err != nil {
t.Fatal(err)
}
if out, why := d.SendEndNotice("c1", "one-more", noticeDay); out != EndNoticeSent {
t.Fatalf("a day later: %s (%s), want sent — the limit delays, it does not drop", out, why)
}
}
// The mail hold and a failed send tell nobody, and say which it was.
func TestNotice_HeldOrFailed_IsNotSent(t *testing.T) {
d, mails := receiptDispatcher(t, "customer@example.com", nil)
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, mailhold.FileName), nil, 0o644); err != nil {
t.Fatal(err)
}
d.SetMailHold(mailhold.New(dir, nil))
if out, why := d.SendEndNotice("c1", "vikunja", noticeDay); out != EndNoticeHeld || !strings.Contains(why, "MAIL-HOLD") || len(*mails) != 0 {
t.Fatalf("held: outcome %s (%q), %d mail(s)", out, why, len(*mails))
}
if !d.MailHeld() {
t.Fatal("MailHeld() = false with the marker present")
}
d.SetMailHold(nil)
d.sendEmailFn = func(string, string, string, map[string]string) error { return errors.New("resend: 503") }
if out, _ := d.SendEndNotice("c1", "vikunja", noticeDay); out != EndNoticeFailed {
t.Fatalf("failed send: outcome %s", out)
}
// neither left a „sent” row: the limit must not count a mail that did not go
if n, _ := d.store.CountCustomerSentSince("c1", EndNoticeEvent, time.Now().Add(-time.Hour)); n != 0 {
t.Fatalf("sent rows after a held and a failed try: %d", n)
}
// no mail service at all
d.resendAPIKey = ""
if out, _ := d.SendEndNotice("c1", "vikunja", noticeDay); out != EndNoticeFailed {
t.Fatalf("no mail service: outcome %s", out)
}
}
// A name that is not an app name never reaches a mail.
func TestNotice_OnlyAnAppNameIsMailed(t *testing.T) {
d, mails := receiptDispatcher(t, "customer@example.com", nil)
for _, app := range []string{"", "pay now at evil.example", "<b>x</b>", "traefik", "_shares", strings.Repeat("a", 65)} {
if out, _ := d.SendEndNotice("c1", app, noticeDay); out == EndNoticeSent || len(*mails) != 0 {
t.Fatalf("%q: outcome %s, %d mail(s)", app, out, len(*mails))
}
}
}
// The receipt register does not open any other type: an ordinary info event still mails nobody, and a type the
// household switched off still stays off.
func TestD84_Receipt_OpensNothingElse(t *testing.T) {
d, mails := receiptDispatcher(t, "customer@example.com", []string{})
d.ProcessBoxEvent("c1", "app_removed", "info", "Alkalmazás eltávolítva.", "", `{"stack_name":"vikunja"}`, "controller")
d.ProcessBoxEvent("c1", "app_stopped_unhealthy", "error", "x", "", `{"stack_name":"vikunja"}`, "controller")
for _, m := range *mails {
if m.to == "customer@example.com" {
t.Fatalf("the household was mailed a type it switched off / an info event: %+v", m)
}
}
if len(householdReceiptEvents) != 1 || !householdReceiptEvents["offsite_end_scheduled"] {
t.Fatalf("the receipt register changed: %v — every entry bypasses the household's switches and needs its own ruling", householdReceiptEvents)
}
if operatorOnlyEvents["offsite_end_scheduled"] {
t.Fatal("a receipt cannot be operator-only")
}
}
// SendOperatorNow answers only when the mail service has: nil = accepted. It refuses a type a household could get.
func TestNotice_SendOperatorNow(t *testing.T) {
d, mails := receiptDispatcher(t, "customer@example.com", nil)
if err := d.SendOperatorNow("c1", "offsite_end_notice_operator", "warning", "m", `{"subject":"vikunja"}`); err != nil || len(*mails) != 1 {
t.Fatalf("err=%v mails=%d", err, len(*mails))
}
if (*mails)[0].to != "operator@felhom.eu" {
t.Fatalf("went to %q", (*mails)[0].to)
}
if err := d.SendOperatorNow("c1", "app_stopped_unhealthy", "warning", "m", ""); err == nil {
t.Fatal("a type that is not operator-only was accepted")
}
d.sendEmailFn = func(string, string, string, map[string]string) error { return errors.New("resend: 503") }
if err := d.SendOperatorNow("c1", "offsite_end_notice_operator", "warning", "m", ""); err == nil {
t.Fatal("a failed send answered nil — the caller would start the 30 days on a mail that never went")
}
d.operatorOn = false
if err := d.SendOperatorNow("c1", "offsite_end_notice_operator", "warning", "m", ""); err == nil {
t.Fatal("the operator's mail is off, and the call answered nil")
}
for _, typ := range []string{"offsite_end_notice_operator", "offsite_end_notice_stuck"} {
if !IsOperatorOnly(typ) {
t.Fatalf("%s must be operator-only", typ)
}
}
if !HasPerSubjectCooldown("offsite_end_notice_stuck") {
t.Fatal("offsite_end_notice_stuck: two apps stuck in one hour must be two mails")
}
}
+2 -8
View File
@@ -96,15 +96,9 @@ func customerMailCases(lang string) []mailCase {
},
mailCase{
name: "customer_shape_offsite_end_scheduled_receipt",
comment: "D84: the household's receipt — the entry names the app, the box's sentence is the line, no raw details",
comment: "D84 + decision 315: the household's notice as the hub sends it — the entry names the app, the hub's sentence is the line, no raw details",
render: func() (string, string) {
box := "vikunja: az alkalmazás távoli másolatai törlésre vannak kijelölve. Törlésük napja: 2026. 12. 15. Addig a Mentések oldalon a „Mégis megtartom” gombbal visszavonhatod."
household := ""
if lang == "en" {
household = "vikunja: the off-site copies of this app are marked for deletion. They are deleted on: 2026-12-15. Until then you can take this back on the Backups page with the “Keep it after all” button."
}
return FormatCustomerEmail(lang, "demo-fixture", "offsite_end_scheduled", "info", box, household,
`{"stack_name":"vikunja","removed_at":"2026-11-15T10:00:00Z","ends_at":"2026-12-15T10:00:00Z","subject":"vikunja|2026-11-15T10:00:00Z"}`)
return FormatReceiptEmail(lang, "demo-fixture", "offsite_end_scheduled", "info", "vikunja", time.Date(2026, 12, 15, 12, 0, 0, 0, time.UTC))
},
},
mailCase{
+1 -1
View File
@@ -482,7 +482,7 @@ var appNamedMailEvents = map[string]bool{
"app_update_held": true,
// v0.123.0 (`09` §3 decision 28): the box stopped an app that kept crashing / ran out of memory.
"app_stopped_unhealthy": true,
// D84: the receipt's headline names the app; the box's own sentence (the day, and how to take it back) follows.
// D84: the receipt's headline names the app; the hub's own sentence (the day, and how to take it back) follows.
"offsite_end_scheduled": true,
}
@@ -1,4 +1,4 @@
# D84: the household's receipt — the entry names the app, the box's sentence is the line, no raw details
# D84 + decision 315: the household's notice as the hub sends it — the entry names the app, the hub's sentence is the line, no raw details
SUBJECT: [Felhom] Information: vikunja: the off-site copies are marked for deletion
---
Dear Customer,
@@ -1,4 +1,4 @@
# D84: the household's receipt — the entry names the app, the box's sentence is the line, no raw details
# D84 + decision 315: the household's notice as the hub sends it — the entry names the app, the hub's sentence is the line, no raw details
SUBJECT: [Felhom] Információ: vikunja: a távoli másolatok törlésre kijelölve
---
Kedves Ügyfél!
+55 -2
View File
@@ -20,6 +20,11 @@ import (
// 1. THE APP. The hub's own record (store.AppPresences, written from the box's reports by the hub's clock) must say
// the app has been absent from that box's app list for EndedAppAfter. An app the newest report lists, an app the
// hub has no record of, an app absent for a shorter time and a reserved name are refused, each with its reason.
// 1b. THE NOTICE (`09` §3 decision 315 — „no notice, no deletion”). The hub's own record (store.EndNotices, written
// only by the hub when its mail service accepted the mail) must say the household — or, for a household with no
// usable address, the operator — was told about THIS press EndedAppAfter ago, by the hub's clock. Beside the 30
// days of absence, not in place of them. No record, a record not told, a record told later: refused, with the
// reason. So an app a box names only in a window request (never in a report) is never agreed.
// 2. THE ID. A snapshot id is agreed only when the hub's own listing of the repository (the 07:10 audit's stored
// baseline — names and upload times, read over the hub's own login) holds it AND shows it uploaded no later than
// the moment the hub first saw the app absent. An app that is gone makes no new snapshots, so a younger file can
@@ -39,6 +44,9 @@ const (
// Review of 2026-11-15 — two more operator warnings, both mailed, both hub-made (no box can POST one):
EventEndedClaimUnseen = "offsite_end_claim_unseen" // a box first names an app the hub never saw installed
EventEndedLargeAgreement = "offsite_ended_large_agreement" // one window was asked for more than half of the store
// The notice lock (`09` §3 decision 315) — two operator warnings, both mailed, both hub-made (api/end_notice.go):
EventEndNoticeOperator = "offsite_end_notice_operator" // told in place of a household with no usable address
EventEndNoticeStuck = "offsite_end_notice_stuck" // a notice has told nobody for three days
// A request for more than half of the store waits largeEndedWait after the window that first saw it; that first
// sight is remembered for largeEndedMemory (a new large request after that waits again).
@@ -67,7 +75,7 @@ func reservedEndedName(app string) bool {
// agreeEnded is the PURE decision. presence is the hub's own record; listing the hub's own stored listing (ok=false
// when there is none). It returns the agreed ids (sorted), the agreed ids per agreed app, and every refusal.
func agreeEnded(now time.Time, presence map[string]store.AppPresence, listing store.OffsiteSnapshotListing, haveListing bool, claims []EndedClaim) (ids []string, byApp map[string][]string, refused []store.EndedRefusal) {
func agreeEnded(now time.Time, presence map[string]store.AppPresence, notices map[string]store.EndNotice, listing store.OffsiteSnapshotListing, haveListing bool, claims []EndedClaim) (ids []string, byApp map[string][]string, refused []store.EndedRefusal) {
if len(claims) > maxEndedClaims {
for _, c := range claims[maxEndedClaims:] {
refused = append(refused, store.EndedRefusal{App: ledgerName(c.App), Reason: fmt.Sprintf("more than %d apps in one request", maxEndedClaims)})
@@ -97,6 +105,9 @@ func agreeEnded(now time.Time, presence map[string]store.AppPresence, listing st
reason = fmt.Sprintf("the hub has seen the app absent since %s — %d full day(s), %d needed",
p.AbsentSince.UTC().Format("2006-01-02 15:04 UTC"), int(now.Sub(p.AbsentSince).Hours()/24), int(EndedAppAfter.Hours()/24))
}
if reason == "" {
reason = untoldReason(now, notices, c.App)
}
if reason != "" {
refusedApp[c.App] = true
// A claimed name that has not the shape of an app name is the box's free text: it is never written
@@ -185,14 +196,47 @@ func agreeEnded(now time.Time, presence map[string]store.AppPresence, listing st
return ids, byApp, refused
}
// untoldReason is the notice lock's half of the decision: "" when the hub's own record says the household (or the
// operator in its place) was told about this app EndedAppAfter ago; else the reason, in the hub's own words.
func untoldReason(now time.Time, notices map[string]store.EndNotice, app string) string {
n, ok := notices[app]
switch {
case !ok:
return reasonNoNotice
case !n.Told():
why := n.LastReason
if why == "" {
why = "the notice has not been sent yet"
}
return "the household has not been told yet, so nothing is removed: " + why
case now.Sub(n.ToldAt) < EndedAppAfter:
who := "the household was"
if n.ToldTo == store.EndNoticeOperator {
who = "the operator (the household has no usable address) was"
}
return fmt.Sprintf("%s told on %s — %d full day(s), %d needed",
who, n.ToldAt.UTC().Format("2006-01-02 15:04 UTC"), int(now.Sub(n.ToldAt).Hours()/24), int(EndedAppAfter.Hours()/24))
}
return ""
}
const reasonNoNotice = "the hub has no record that the household was told: no report of the box names this app in a pending end record"
// clearDisagreement: a refusal the operator must hear about. A box that asks a little early (its clock is minutes or
// hours ahead of the hub's first sight of the absence) is refused quietly and asks again next week; a box that claims
// an end for an app it reports as installed, or that the hub has seen absent for less than 29 days, is not a matter of
// minutes — its clock is wrong by a day or more (R-1384), or it is not telling the truth.
//
// The notice lock adds one clear case: the box asks for an app whose absence is old enough, and NO report of it ever
// named the app in a pending end record (an honest box names it in every report). A notice that is only not sent
// yet, or not 30 days old, is quiet here — the operator's line for that is `offsite_end_notice_stuck`.
func clearDisagreement(now time.Time, presence map[string]store.AppPresence, r store.EndedRefusal) bool {
if r.App == "" {
return false
}
if r.Reason == reasonNoNotice {
return true
}
p, known := presence[r.App]
// An installed app has no absence date (store.AppPresences), so the zero date covers it too.
if !known || p.AbsentSince.IsZero() {
@@ -216,8 +260,17 @@ func (s *Service) agreeEndedFor(customerID string, windowID int64, maxRemove int
}
return nil, nil, refused
}
// The notice lock: a record that cannot be read is „nobody was told” — nothing agreed.
notices, nerr := s.Store.EndNotices(customerID)
if nerr != nil {
s.logf("[WARN] offsitekeys: ended apps %s: the hub's own notice record could not be read (%v) — nothing agreed", customerID, nerr)
for _, c := range claims {
refused = append(refused, store.EndedRefusal{App: ledgerName(c.App), Reason: "the hub's own notice record could not be read"})
}
return nil, nil, refused
}
listing, have := s.Store.GetOffsiteSnapshotListing(customerID)
ids, byApp, refused = agreeEnded(now, presence, listing, have, claims)
ids, byApp, refused = agreeEnded(now, presence, notices, listing, have, claims)
// Review of 2026-11-15 — what bounds a box that invents an ended app, or stops listing a living one, and 30 days
// later names much of the store under it:
//
@@ -38,6 +38,7 @@ func ghostFixture(t *testing.T) (*Service, *[]evt, []string) {
if err := s.Store.NoteAppList("c1", []string{"immich"}, []string{"ghost"}, now.Add(-time.Hour)); err != nil {
t.Fatal(err)
}
toldLongAgo(t, s, "ghost")
*events = nil
return s, events, all
}
@@ -133,6 +134,7 @@ func TestD84_ALargeRequestForOtherAppsWaitsItsOwnWeek(t *testing.T) {
s, events, all := ghostFixture(t)
now := time.Now().UTC()
_ = s.Store.NoteAppList("c1", []string{"immich"}, []string{"ghost", "second"}, now.Add(-31*24*time.Hour-time.Minute))
toldLongAgo(t, s, "second")
g0 := openGranted(t, s, 20, EndedClaim{App: "ghost", SnapshotIDs: all})
_ = s.CloseWindowFor(context.Background(), "c1", WindowResult{WindowID: g0.WindowID, CountBefore: 20, CountAfter: 20, Outcome: "nothing"})
aWeekLater(t, s, g0.WindowID)
@@ -96,6 +96,7 @@ func TestD84_TwoAppSetsThatDifferAfterTheTenthNameDoNotShareAWait(t *testing.T)
if err := s.Store.NoteAppList("c1", []string{"immich"}, names, now.Add(-31*24*time.Hour)); err != nil {
t.Fatal(err)
}
toldLongAgo(t, s, names...)
set := func(last string) []EndedClaim {
var cl []EndedClaim
for i := 0; i < 10; i++ {
+20 -1
View File
@@ -105,7 +105,12 @@ func TestD84_AgreeEnded_TheHubsOwnRecordsDecide(t *testing.T) {
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
ids, byApp, refused := agreeEnded(now, c.presence, listing, c.listing, c.claims)
// the notice lock is not this table's subject: every claimed app was told 31 days ago (TestNotice_* pin it)
notices := map[string]store.EndNotice{}
for _, cl := range c.claims {
notices[cl.App] = store.EndNotice{App: cl.App, ToldAt: days(31), ToldTo: store.EndNoticeHousehold}
}
ids, byApp, refused := agreeEnded(now, c.presence, notices, listing, c.listing, c.claims)
if strings.Join(ids, ",") != strings.Join(c.wantIDs, ",") {
t.Fatalf("agreed ids = %v, want %v (refused: %+v)", shortIDs(ids), shortIDs(c.wantIDs), refused)
}
@@ -147,9 +152,22 @@ func endedFixture(t *testing.T) (*Service, *[]evt) {
if err := s.Store.NoteAppList("c1", []string{"immich"}, nil, now.Add(-31*24*time.Hour)); err != nil {
t.Fatal(err)
}
toldLongAgo(t, s, "vikunja")
return s, events
}
// toldLongAgo writes the hub's own notice record for apps as told to the household 31 days ago (the notice lock is
// pinned by TestNotice_*; the D84 tests are about the other conditions).
func toldLongAgo(t *testing.T, s *Service, apps ...string) {
t.Helper()
at := time.Now().UTC().Add(-31 * 24 * time.Hour)
for _, app := range apps {
if err := s.Store.SetEndNoticeForTest("c1", app, "", at, at, store.EndNoticeHousehold); err != nil {
t.Fatal(err)
}
}
}
func openGranted(t *testing.T, s *Service, count int, ended ...EndedClaim) WindowGrant {
t.Helper()
if err := s.Store.GrantOffsiteWindowOnce("c1"); err != nil {
@@ -231,6 +249,7 @@ func TestD84_Window_ARefusalTellsTheOperatorOnlyWhenItIsClear(t *testing.T) {
_ = s.Store.NoteAppList("c1", []string{"immich", "kimai", "nearly"}, nil, now.Add(-80*24*time.Hour))
_ = s.Store.NoteAppList("c1", []string{"immich", "kimai"}, nil, now.Add(-EndedAppAfter+time.Hour))
_ = s.Store.NoteAppList("c1", []string{"immich"}, nil, now.Add(-2*24*time.Hour))
toldLongAgo(t, s, "kimai", "nearly", "immich")
g := openGranted(t, s, 20,
EndedClaim{App: "kimai", EndsAt: now.Add(-time.Hour).Format(time.RFC3339), SnapshotIDs: []string{eid(1)}},
EndedClaim{App: "nearly", SnapshotIDs: []string{eid(2)}},
@@ -0,0 +1,149 @@
package offsitekeys
import (
"strings"
"testing"
"time"
"gitea.dooplex.hu/admin/felhom-hub/internal/store"
)
// THE NOTICE LOCK, the agreement's half (`09` §3 decision 315; `07-backup-architecture.md` §8.2f): the hub agrees to
// an app only when ITS OWN record says the household was told at least 30 days ago — beside the 30 days of absence.
// RED-PROOFS: documentation/audits/build-release-4b-2026-11-15/A/redproof-notice-agree.txt.
// The pure decision. In every row the app has been absent for 40 days and its file is 60 days old: without the
// notice lock every row would be agreed.
func TestNotice_AgreeEnded_NoNoticeNoDeletion(t *testing.T) {
now := time.Date(2026, 12, 20, 3, 0, 0, 0, time.UTC)
days := func(n int) time.Time { return now.Add(-time.Duration(n) * 24 * time.Hour) }
listing := store.OffsiteSnapshotListing{ListedAt: now.Add(-20 * time.Hour), Entries: map[string]time.Time{eid(1): days(60), eid(2): days(60)}}
absent40 := map[string]store.AppPresence{"vikunja": {AbsentSince: days(40)}, "kimai": {AbsentSince: days(40)}}
claim := []EndedClaim{{App: "vikunja", RemovedAt: days(400).Format(time.RFC3339), EndsAt: days(370).Format(time.RFC3339), SnapshotIDs: []string{eid(1)}}}
told := func(at time.Time, to string) map[string]store.EndNotice {
return map[string]store.EndNotice{"vikunja": {App: "vikunja", ToldAt: at, ToldTo: to}}
}
cases := []struct {
name string
presence map[string]store.AppPresence
notices map[string]store.EndNotice
claims []EndedClaim
wantIDs []string
refused string // a fragment of vikunja's reason
}{
{name: "told 30 days and a minute ago: agreed", presence: absent40, notices: told(now.Add(-EndedAppAfter-time.Minute), store.EndNoticeHousehold), claims: claim, wantIDs: []string{eid(1)}},
{name: "the operator was told in the household's place 31 days ago: agreed", presence: absent40, notices: told(days(31), store.EndNoticeOperator), claims: claim, wantIDs: []string{eid(1)}},
{name: "no notice record at all (the box names the app only in the window request)", presence: absent40, notices: nil, claims: claim, refused: "no record that the household was told"},
{name: "a record, and nobody told yet (the send failed)", presence: absent40,
notices: map[string]store.EndNotice{"vikunja": {App: "vikunja", LastReason: "the mail service did not accept the mail"}}, claims: claim,
refused: "has not been told yet, so nothing is removed: the mail service did not accept the mail"},
{name: "a record, and nobody told yet (no try so far)", presence: absent40,
notices: map[string]store.EndNotice{"vikunja": {App: "vikunja"}}, claims: claim, refused: "has not been told yet"},
{name: "told 29 days and 23 hours ago", presence: absent40, notices: told(now.Add(-EndedAppAfter+time.Hour), store.EndNoticeHousehold), claims: claim, refused: "29 full day(s), 30 needed"},
{name: "told yesterday (the mail waited for the daily limit)", presence: absent40, notices: told(days(1), store.EndNoticeHousehold), claims: claim, refused: "the household was told on"},
{name: "the operator was told 5 days ago", presence: absent40, notices: told(days(5), store.EndNoticeOperator), claims: claim, refused: "the operator (the household has no usable address) was told on"},
{name: "a told time and no recipient (a value the hub never writes)", presence: absent40, notices: told(days(90), ""), claims: claim, refused: "has not been told yet"},
{name: "BESIDE the absence, not in its place: told 40 days ago, absent for 10", presence: map[string]store.AppPresence{"vikunja": {AbsentSince: days(10)}},
notices: told(days(40), store.EndNoticeHousehold), claims: claim, refused: "10 full day(s), 30 needed"},
{name: "told, and installed again", presence: map[string]store.AppPresence{"vikunja": {Present: true}},
notices: told(days(40), store.EndNoticeHousehold), claims: claim, refused: "lists the app as installed"},
{name: "a snapshot listed under a told app AND an app nobody was told about stays", presence: absent40,
notices: told(days(40), store.EndNoticeHousehold),
claims: []EndedClaim{{App: "vikunja", SnapshotIDs: []string{eid(1), eid(2)}}, {App: "kimai", SnapshotIDs: []string{eid(2)}}},
wantIDs: []string{eid(1)}},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
ids, _, refused := agreeEnded(now, c.presence, c.notices, listing, true, c.claims)
if strings.Join(ids, ",") != strings.Join(c.wantIDs, ",") {
t.Fatalf("agreed ids = %v, want %v (refused: %+v)", shortIDs(ids), shortIDs(c.wantIDs), refused)
}
if got := endedRefusedFor(refused, "vikunja"); !strings.Contains(got, c.refused) || (c.refused == "" && got != "") {
t.Fatalf("vikunja's refusal = %q, want it to say %q", got, c.refused)
}
})
}
}
// Through the window: the hub's own app record says vikunja left 31 days ago and its files are old — and the answer
// depends on the notice record alone.
func TestNotice_Window_TheNoticeRecordDecides(t *testing.T) {
ids := []string{eid(0), eid(1)}
claim := EndedClaim{App: "vikunja", SnapshotIDs: ids}
now := time.Now().UTC()
// (a) the box names the app ONLY in the window request, never in a report: no record → nothing, and the operator
// hears it (an honest box names a pending record in every report).
s, events := endedFixtureNoNotice(t)
g := openGranted(t, s, 20, claim)
if len(g.EndedAllowed) != 0 || !hasReason(g.EndedRefused, "no record that the household was told") {
t.Fatalf("no notice record: agreed %v, refused %+v", shortIDs(g.EndedAllowed), g.EndedRefused)
}
if w, _ := s.Store.GetOffsiteWindow(g.WindowID); len(w.EndedAgreed) != 0 || len(w.EndedRefused) == 0 {
t.Fatalf("the window's own row: %+v", w)
}
if e := ofType(*events, EventEndedAppRefused); len(e) != 1 || !strings.Contains(e[0].msg, "no record that the household was told") {
t.Fatalf("the operator must hear of a box that asks for an app it never named in a report: %+v", *events)
}
// (b) the notice could not go out (the mail hold): nothing, the answer names the reason, and this refusal is
// quiet (the operator's line for it is offsite_end_notice_stuck).
s, events = endedFixtureNoNotice(t)
first := now.Add(-31 * 24 * time.Hour)
if err := s.Store.SetEndNoticeForTest("c1", "vikunja", "", first, time.Time{}, ""); err != nil {
t.Fatal(err)
}
if err := s.Store.MarkEndNoticeTried("c1", "vikunja", "", "the hub is not sending mail (the MAIL-HOLD marker is present)", now); err != nil {
t.Fatal(err)
}
g = openGranted(t, s, 20, claim)
if len(g.EndedAllowed) != 0 || !hasReason(g.EndedRefused, "MAIL-HOLD") {
t.Fatalf("a notice that could not go out: agreed %v, refused %+v", shortIDs(g.EndedAllowed), g.EndedRefused)
}
if e := ofType(*events, EventEndedAppRefused); len(e) != 0 {
t.Fatalf("a notice that is only not sent yet raised the disagreement warning: %+v", e)
}
// (c) told 29 days ago: nothing. (d) told 31 days ago: agreed.
s, _ = endedFixtureNoNotice(t)
at := now.Add(-29 * 24 * time.Hour)
if err := s.Store.SetEndNoticeForTest("c1", "vikunja", "", at, at, store.EndNoticeHousehold); err != nil {
t.Fatal(err)
}
if g := openGranted(t, s, 20, claim); len(g.EndedAllowed) != 0 || !hasReason(g.EndedRefused, "29 full day(s), 30 needed") {
t.Fatalf("told 29 days ago: agreed %v, refused %+v", shortIDs(g.EndedAllowed), g.EndedRefused)
}
s, _ = endedFixtureNoNotice(t)
at = now.Add(-31 * 24 * time.Hour)
if err := s.Store.SetEndNoticeForTest("c1", "vikunja", "", at, at, store.EndNoticeHousehold); err != nil {
t.Fatal(err)
}
if g := openGranted(t, s, 20, claim); strings.Join(g.EndedAllowed, ",") != strings.Join(ids, ",") {
t.Fatalf("positive control — told 31 days ago, absent 31 days: agreed %v, refused %+v", shortIDs(g.EndedAllowed), g.EndedRefused)
}
// (e) the household is deleted between the notice and the window: its record is gone, nothing is agreed.
s, _ = endedFixtureNoNotice(t)
if err := s.Store.SetEndNoticeForTest("c1", "vikunja", "", at, at, store.EndNoticeHousehold); err != nil {
t.Fatal(err)
}
if err := s.Store.PurgeCustomerResidue("c1"); err != nil {
t.Fatal(err)
}
if ids, _, refused := s.agreeEndedFor("c1", 1, 10, []EndedClaim{claim}); len(ids) != 0 || len(refused) == 0 {
t.Fatalf("a deleted household: agreed %v, refused %+v", shortIDs(ids), refused)
}
}
// endedFixtureNoNotice is endedFixture without the notice row it writes.
func endedFixtureNoNotice(t *testing.T) (*Service, *[]evt) {
t.Helper()
s, events := endedFixture(t)
if err := s.Store.NoteEndNotices("c1", []string{"vikunja"}, nil, true, time.Now()); err != nil { // an install voids it
t.Fatal(err)
}
if ns, _ := s.Store.EndNotices("c1"); len(ns) != 0 {
t.Fatalf("fixture: %+v", ns)
}
return s, events
}
+4 -1
View File
@@ -55,6 +55,8 @@ type CustomerResidue struct {
OffsiteAppPresence int
// The hub's own first sight of a large ended-app request (offsite_ended.go).
OffsiteLargeRequests int
// The notice lock: the hub's own record that the household was told (offsite_end_notice.go).
OffsiteEndNotices int
}
// Total is the single number the "is there anything left at all?" decision hangs on.
@@ -64,7 +66,7 @@ func (r *CustomerResidue) Total() int {
}
return r.Reports + r.AppTelemetry + r.AppLogTails + r.LogTailRequests +
r.NotificationPrefs + r.SelfBindTokens + r.ApplianceRegistrations + r.AppLogIssueRefs +
r.OffsiteKeys + r.OffsiteWindows + r.OffsiteAbandonRequests + r.OffsiteAppPresence + r.OffsiteLargeRequests
r.OffsiteKeys + r.OffsiteWindows + r.OffsiteAbandonRequests + r.OffsiteAppPresence + r.OffsiteLargeRequests + r.OffsiteEndNotices
}
// residueQueries is the ONE list both the counter and the purge walk, so a table can never be
@@ -85,6 +87,7 @@ var residueQueries = []struct {
{"offsite_abandon_requests", func(r *CustomerResidue) *int { return &r.OffsiteAbandonRequests }},
{"offsite_app_presence", func(r *CustomerResidue) *int { return &r.OffsiteAppPresence }},
{"offsite_large_requests", func(r *CustomerResidue) *int { return &r.OffsiteLargeRequests }},
{"offsite_end_notices", func(r *CustomerResidue) *int { return &r.OffsiteEndNotices }},
}
// CustomerResidue counts the customer's report-derived + binding residue. Read-only.
+300
View File
@@ -0,0 +1,300 @@
package store
import (
"database/sql"
"fmt"
"sort"
"sync"
"time"
)
// THE NOTICE LOCK (`09` §3 decision 315; `07-backup-architecture.md` §8.2f „No notice, no deletion”) — THE HUB'S OWN
// RECORD that a household was told its app's off-site copies will be deleted.
//
// One row per (household, app) that a box's REPORT names in a pending off-site end record. The hub mails the notice
// itself (api.endNotices → notify.SendEndNotice) and writes `told_at` — by ITS clock — only when the mail service
// accepted the mail. offsitekeys.agreeEnded agrees to an app only when this table says it was told EndedAppAfter ago.
//
// Why a table and not the sent log (`notification_log`): the 30 days are a condition of a deletion, so they must not
// rest on a log that is pruned (deleted households today; anything tomorrow), nor on anything a box says. Nothing
// here is written from a box's date: `press` is the box's `removed_at`, kept ONLY to tell one press from the next —
// a different value VOIDS the notice (the safe direction), and no value can make one.
//
// A row is deleted — and with it the notice, so a new press needs a new notice and a new 30 days — when:
// - a report lists the app as installed (an install);
// - a report that shows the off-site tier ON no longer names the app (a take-back, or the window has run);
// - the household goes (residueQueries, orphanTables).
//
// A row is RESET (told_at cleared, first_named = now) when a report names the app with another `press`.
// A report that does not show the tier (it is off, or waits for a key) changes nothing: its silence is not a
// take-back. A database put back from a copy that lacks a row has no notice: the hub tells again and waits again.
// maxEndNoticeRows bounds one household's rows (a box's list is the box's word). A claim above it is not stored, so
// it is never told and never agreed.
const maxEndNoticeRows = 50
// EndNoticeHousehold / EndNoticeOperator: who the notice that counts went to.
const (
EndNoticeHousehold = "household"
EndNoticeOperator = "operator"
)
func (s *Store) migrateOffsiteEndNotices() error {
if _, err := s.db.Exec(`
CREATE TABLE IF NOT EXISTS offsite_end_notices (
customer_id TEXT NOT NULL,
app TEXT NOT NULL,
press TEXT NOT NULL DEFAULT '',
first_named TEXT NOT NULL,
told_at TEXT,
told_to TEXT NOT NULL DEFAULT '',
tries INTEGER NOT NULL DEFAULT 0,
last_try TEXT,
last_reason TEXT NOT NULL DEFAULT '',
stuck_told TEXT,
PRIMARY KEY (customer_id, app)
);`); err != nil {
return fmt.Errorf("offsite_end_notices: %w", err)
}
return nil
}
// EndClaim is one pending end record of a report, as far as the notice needs it.
type EndClaim struct {
App string
// Press is the box's `removed_at`. NormalizeEndPress decides what is stored of it.
Press string
}
// EndNotice is the hub's record for one app of one household.
type EndNotice struct {
App string
Press string
// FirstNamed: the hub's time of the first report that named this press.
FirstNamed time.Time
// ToldAt: the hub's time at which the notice that counts was accepted by the mail service. Zero = not told.
ToldAt time.Time
// ToldTo: EndNoticeHousehold, or EndNoticeOperator for a household that had no usable address.
ToldTo string
Tries int
LastTry time.Time
LastReason string // the hub's own words for why the last try did not tell anybody
// StuckTold: the hub's time of the operator's one line about a notice that could not go out for days.
StuckTold time.Time
}
// Told: the hub's record says somebody was told.
func (n EndNotice) Told() bool { return !n.ToldAt.IsZero() && n.ToldTo != "" }
// NormalizeEndPress turns a box's `removed_at` into the key of a press: the same instant in UTC when it is a time,
// else the empty key. It is never read as a date.
func NormalizeEndPress(removedAt string) string {
if len(removedAt) > 40 {
return ""
}
t, err := time.Parse(time.RFC3339, removedAt)
if err != nil {
return ""
}
return t.UTC().Format(time.RFC3339)
}
// NoteEndNotices updates the record from ONE report (see the header). deployed is the report's app list; claims are
// its pending end records; tierOn says the report shows the off-site tier switched on (only then is a missing claim
// a take-back). now is the HUB's clock.
func (s *Store) NoteEndNotices(customerID string, deployed []string, claims []EndClaim, tierOn bool, now time.Time) error {
if customerID == "" {
return fmt.Errorf("NoteEndNotices: empty customer_id")
}
if len(deployed) > maxPresenceApps {
deployed = deployed[:maxPresenceApps]
}
if len(claims) > maxPresenceApps {
claims = claims[:maxPresenceApps]
}
cur, err := s.EndNotices(customerID)
if err != nil {
return err
}
if len(cur) == 0 && len(claims) == 0 {
return nil // the steady box: one read
}
in := make(map[string]bool, len(deployed))
for _, app := range deployed {
in[app] = true
}
claimed := map[string]string{} // app → press (the first claim of a name wins)
var order []string
for _, c := range claims {
if !ValidPresenceApp(c.App) || ReservedPresenceApp(c.App) || in[c.App] {
continue
}
if _, dup := claimed[c.App]; dup {
continue
}
claimed[c.App] = NormalizeEndPress(c.Press)
order = append(order, c.App)
}
var drop, fresh []string
for app, n := range cur {
press, named := claimed[app]
switch {
case in[app]: // an install voids the notice
drop = append(drop, app)
case named && press != n.Press: // another press: a new notice, a new 30 days
drop = append(drop, app)
fresh = append(fresh, app)
case !named && tierOn: // a take-back, or the window has run
drop = append(drop, app)
}
}
rows := len(cur) - len(drop)
notStored := 0
for _, app := range order {
if _, ok := cur[app]; ok {
continue
}
fresh = append(fresh, app)
}
sort.Strings(fresh)
if over := rows + len(fresh) - maxEndNoticeRows; over > 0 {
notStored = over
fresh = fresh[:len(fresh)-over]
}
s.noteEndNoticesFull(customerID, notStored)
if len(drop) == 0 && len(fresh) == 0 {
return nil
}
stamp := now.UTC().Format(time.RFC3339)
tx, err := s.db.Begin()
if err != nil {
return err
}
defer tx.Rollback()
for _, app := range drop {
if _, err := tx.Exec(`DELETE FROM offsite_end_notices WHERE customer_id = ? AND app = ?`, customerID, app); err != nil {
return fmt.Errorf("NoteEndNotices %s/%s: %w", customerID, app, err)
}
}
for _, app := range fresh {
if _, err := tx.Exec(`INSERT INTO offsite_end_notices (customer_id, app, press, first_named) VALUES (?, ?, ?, ?)`,
customerID, app, claimed[app], stamp); err != nil {
return fmt.Errorf("NoteEndNotices %s/%s: %w", customerID, app, err)
}
}
return tx.Commit()
}
// endNoticesFull: the row limit is said in the log once per household, not once per report (in memory).
var (
endNoticesFullMu sync.Mutex
endNoticesFull = map[string]bool{}
)
func (s *Store) noteEndNoticesFull(customerID string, notStored int) {
endNoticesFullMu.Lock()
defer endNoticesFullMu.Unlock()
if notStored == 0 {
delete(endNoticesFull, customerID)
return
}
if endNoticesFull[customerID] {
return
}
endNoticesFull[customerID] = true
if s.logger != nil {
s.logger.Printf("[WARN] off-site end notices for %s are at their limit of %d apps: %d pending end record(s) of this report were NOT stored — nobody is told about them and the hub agrees to none of them. A household does not remove that many apps at once — look at what the box reports. (Said once; not repeated per report.)",
customerID, maxEndNoticeRows, notStored)
}
}
// EndNotices returns the hub's record for one household, by app. A date that cannot be read is the zero time — and
// a zero ToldAt is „not told”.
func (s *Store) EndNotices(customerID string) (map[string]EndNotice, error) {
rows, err := s.db.Query(`SELECT app, press, first_named, told_at, told_to, tries, last_try, last_reason, stuck_told
FROM offsite_end_notices WHERE customer_id = ?`, customerID)
if err != nil {
return nil, err
}
defer rows.Close()
out := map[string]EndNotice{}
for rows.Next() {
var n EndNotice
var first string
var told, lastTry, stuck sql.NullString
if err := rows.Scan(&n.App, &n.Press, &first, &told, &n.ToldTo, &n.Tries, &lastTry, &n.LastReason, &stuck); err != nil {
return nil, err
}
n.FirstNamed = parseNoticeTime(first)
n.ToldAt = parseNoticeTime(told.String)
n.LastTry = parseNoticeTime(lastTry.String)
n.StuckTold = parseNoticeTime(stuck.String)
if n.ToldTo != EndNoticeHousehold && n.ToldTo != EndNoticeOperator {
n.ToldTo, n.ToldAt = "", time.Time{} // a value the hub never writes is „not told”
}
out[n.App] = n
}
return out, rows.Err()
}
func parseNoticeTime(v string) time.Time {
if v == "" {
return time.Time{}
}
t, err := time.Parse(time.RFC3339, v)
if err != nil {
return time.Time{}
}
return t
}
// MarkEndNoticeTold writes that the notice of THIS press was accepted by the mail service, at the hub's time `now`,
// to `to` (EndNoticeHousehold or EndNoticeOperator). It matches the press: a row that was voided or reset while the
// mail was being sent is not marked (ok=false) — that press was never told.
func (s *Store) MarkEndNoticeTold(customerID, app, press, to string, now time.Time) (ok bool, err error) {
if to != EndNoticeHousehold && to != EndNoticeOperator {
return false, fmt.Errorf("MarkEndNoticeTold: unknown recipient %q", to)
}
stamp := now.UTC().Format(time.RFC3339)
res, err := s.db.Exec(`UPDATE offsite_end_notices SET told_at = ?, told_to = ?, tries = tries + 1, last_try = ?, last_reason = ''
WHERE customer_id = ? AND app = ? AND press = ?`, stamp, to, stamp, customerID, app, press)
if err != nil {
return false, err
}
n, _ := res.RowsAffected()
return n == 1, nil
}
// MarkEndNoticeTried writes a try that told nobody, with the hub's own words for why. It never touches told_at.
func (s *Store) MarkEndNoticeTried(customerID, app, press, reason string, now time.Time) error {
_, err := s.db.Exec(`UPDATE offsite_end_notices SET tries = tries + 1, last_try = ?, last_reason = ?
WHERE customer_id = ? AND app = ? AND press = ?`, now.UTC().Format(time.RFC3339), reason, customerID, app, press)
return err
}
// MarkEndNoticeStuckTold writes that the operator's one line about a notice that cannot go out was filed.
func (s *Store) MarkEndNoticeStuckTold(customerID, app, press string, now time.Time) error {
_, err := s.db.Exec(`UPDATE offsite_end_notices SET stuck_told = ? WHERE customer_id = ? AND app = ? AND press = ?`,
now.UTC().Format(time.RFC3339), customerID, app, press)
return err
}
// SetEndNoticeForTest writes a row as if the hub had named the app at firstNamed and (toldAt non-zero) told `to`
// at toldAt. TEST-ONLY.
func (s *Store) SetEndNoticeForTest(customerID, app, press string, firstNamed, toldAt time.Time, to string) error {
var told sql.NullString
if !toldAt.IsZero() {
told = sql.NullString{String: toldAt.UTC().Format(time.RFC3339), Valid: true}
}
_, err := s.db.Exec(`INSERT INTO offsite_end_notices (customer_id, app, press, first_named, told_at, told_to) VALUES (?, ?, ?, ?, ?, ?)
ON CONFLICT(customer_id, app) DO UPDATE SET press = excluded.press, first_named = excluded.first_named, told_at = excluded.told_at, told_to = excluded.told_to`,
customerID, app, press, firstNamed.UTC().Format(time.RFC3339), told, to)
return err
}
// BackdateNotificationLogForTest moves a household's sent-log rows into the past. TEST-ONLY.
func (s *Store) BackdateNotificationLogForTest(customerID string, by time.Duration) error {
_, err := s.db.Exec(`UPDATE notification_log SET created_at = datetime(created_at, ?) WHERE customer_id = ?`,
fmt.Sprintf("-%d seconds", int(by.Seconds())), customerID)
return err
}
@@ -0,0 +1,186 @@
package store
import (
"fmt"
"strings"
"testing"
"time"
)
// THE NOTICE LOCK, the hub's own record (`09` §3 decision 315; `07-backup-architecture.md` §8.2f).
// RED-PROOFS: documentation/audits/build-release-4b-2026-11-15/A/redproof-notice-store.txt.
func endNotice(t *testing.T, s *Store, cust, app string) (EndNotice, bool) {
t.Helper()
ns, err := s.EndNotices(cust)
if err != nil {
t.Fatal(err)
}
n, ok := ns[app]
return n, ok
}
// One press is one row, made at the first report that names it — by the HUB's clock — and nothing a later report
// says moves it. „Told” is written only by the hub, and only for the press it mailed about.
func TestNotice_Record_OnePressOneRow(t *testing.T) {
s := newTestStore(t)
t0 := time.Date(2026, 11, 15, 10, 0, 0, 0, time.UTC)
press := "2026-11-15T09:59:00Z"
note := func(at time.Time, deployed []string, tierOn bool, claims ...EndClaim) {
t.Helper()
if err := s.NoteEndNotices("c1", deployed, claims, tierOn, at); err != nil {
t.Fatal(err)
}
}
note(t0, []string{"immich"}, true, EndClaim{App: "vikunja", Press: press})
n, ok := endNotice(t, s, "c1", "vikunja")
if !ok || n.Told() || !n.FirstNamed.Equal(t0) || n.Press != press {
t.Fatalf("first report: %+v (found=%v) — want one row, not told, first named at the hub's time", n, ok)
}
// the same press in another time zone's writing, a day later: the same row
note(t0.Add(24*time.Hour), []string{"immich"}, true, EndClaim{App: "vikunja", Press: "2026-11-15T10:59:00+01:00"})
if n, _ := endNotice(t, s, "c1", "vikunja"); !n.FirstNamed.Equal(t0) {
t.Fatalf("a later report of the same press moved the row: %+v", n)
}
// told: only with the press the hub mailed about
if ok, err := s.MarkEndNoticeTold("c1", "vikunja", "2026-11-01T00:00:00Z", EndNoticeHousehold, t0); err != nil || ok {
t.Fatalf("a told mark for ANOTHER press was written (ok=%v err=%v)", ok, err)
}
if n, _ := endNotice(t, s, "c1", "vikunja"); n.Told() {
t.Fatalf("told by a mark for another press: %+v", n)
}
if _, err := s.MarkEndNoticeTold("c1", "vikunja", press, "the box", t0); err == nil {
t.Fatal("an unknown recipient was accepted")
}
told := t0.Add(time.Hour)
if ok, err := s.MarkEndNoticeTold("c1", "vikunja", press, EndNoticeHousehold, told); err != nil || !ok {
t.Fatalf("MarkEndNoticeTold: ok=%v err=%v", ok, err)
}
n, _ = endNotice(t, s, "c1", "vikunja")
if !n.Told() || !n.ToldAt.Equal(told) || n.ToldTo != EndNoticeHousehold {
t.Fatalf("told: %+v", n)
}
// a try that told nobody never touches the told time
if err := s.MarkEndNoticeTried("c1", "vikunja", press, "x", told.Add(time.Hour)); err != nil {
t.Fatal(err)
}
if n, _ := endNotice(t, s, "c1", "vikunja"); !n.ToldAt.Equal(told) {
t.Fatalf("a failed try moved the told time: %+v", n)
}
}
// What voids a notice, and what does not.
func TestNotice_Record_WhatVoidsIt(t *testing.T) {
t0 := time.Date(2026, 11, 15, 10, 0, 0, 0, time.UTC)
press := "2026-11-15T09:59:00Z"
cases := []struct {
name string
deployed []string
tierOn bool
claims []EndClaim
want string // "told" (kept), "gone", "fresh" (a new row, not told)
}{
{"the same press again", []string{"immich"}, true, []EndClaim{{App: "vikunja", Press: press}}, "told"},
{"a take-back: the tier is on and no record names the app", []string{"immich"}, true, nil, "gone"},
{"an install: the app is in the app list", []string{"immich", "vikunja"}, true, nil, "gone"},
{"an install that the box still claims (it lies, or it is slow)", []string{"immich", "vikunja"}, true, []EndClaim{{App: "vikunja", Press: press}}, "gone"},
{"another press", []string{"immich"}, true, []EndClaim{{App: "vikunja", Press: "2026-11-15T16:00:00Z"}}, "fresh"},
{"a press whose key is not a time", []string{"immich"}, true, []EndClaim{{App: "vikunja", Press: "yesterday"}}, "fresh"},
{"the report does not show the tier: its silence is not a take-back", []string{"immich"}, false, nil, "told"},
{"the report does not show the tier, and the app is installed", []string{"vikunja"}, false, nil, "gone"},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
s := newTestStore(t)
if err := s.SetEndNoticeForTest("c1", "vikunja", press, t0, t0, EndNoticeHousehold); err != nil {
t.Fatal(err)
}
later := t0.Add(5 * 24 * time.Hour)
if err := s.NoteEndNotices("c1", c.deployed, c.claims, c.tierOn, later); err != nil {
t.Fatal(err)
}
n, ok := endNotice(t, s, "c1", "vikunja")
got := "gone"
switch {
case ok && n.Told() && n.ToldAt.Equal(t0):
got = "told"
case ok && !n.Told() && n.FirstNamed.Equal(later):
got = "fresh"
case ok:
got = fmt.Sprintf("?? %+v", n)
}
if got != c.want {
t.Fatalf("the notice is %s, want %s", got, c.want)
}
})
}
}
// A box's list is the box's word: only app names are stored, at most maxEndNoticeRows per household; a stored value
// the hub never writes is „not told”; a told time that cannot be read is „not told”.
func TestNotice_Record_IsBoundedAndFailsClosed(t *testing.T) {
s := newTestStore(t)
now := time.Date(2026, 11, 15, 10, 0, 0, 0, time.UTC)
var claims []EndClaim
for _, bad := range []string{"", "_shares", "traefik", "felhom-offbox", "<b>x</b>", "pay now", strings.Repeat("a", 65)} {
claims = append(claims, EndClaim{App: bad})
}
for i := 0; i < 80; i++ {
claims = append(claims, EndClaim{App: fmt.Sprintf("app-%02d", i)})
}
if err := s.NoteEndNotices("c1", nil, claims, true, now); err != nil {
t.Fatal(err)
}
ns, _ := s.EndNotices("c1")
if len(ns) != maxEndNoticeRows {
t.Fatalf("rows = %d, want the limit %d", len(ns), maxEndNoticeRows)
}
for app := range ns {
if !strings.HasPrefix(app, "app-") {
t.Fatalf("a name that is not an app name was stored: %q", app)
}
}
// the same 80 again: still the limit, and the rows it holds are not replaced
if err := s.NoteEndNotices("c1", nil, claims, true, now.Add(time.Hour)); err != nil {
t.Fatal(err)
}
ns2, _ := s.EndNotices("c1")
if len(ns2) != maxEndNoticeRows || !ns2["app-00"].FirstNamed.Equal(now) {
t.Fatalf("a second report changed the bounded record: %d rows, app-00 %+v", len(ns2), ns2["app-00"])
}
// fail closed: a recipient the hub never writes, and a told time that is not a time
if _, err := s.db.Exec(`UPDATE offsite_end_notices SET told_at = '2020-01-01T00:00:00Z', told_to = 'box' WHERE app = 'app-00'`); err != nil {
t.Fatal(err)
}
if _, err := s.db.Exec(`UPDATE offsite_end_notices SET told_at = 'long ago', told_to = 'household' WHERE app = 'app-01'`); err != nil {
t.Fatal(err)
}
ns3, _ := s.EndNotices("c1")
if ns3["app-00"].Told() || ns3["app-01"].Told() {
t.Fatalf("an unreadable record counts as told: %+v %+v", ns3["app-00"], ns3["app-01"])
}
}
// The notice record goes with the household (D73's rule for every household-keyed table).
func TestNotice_Record_GoesWithTheHousehold(t *testing.T) {
s := newTestStore(t)
now := time.Now()
if err := s.SetEndNoticeForTest("gone", "vikunja", "", now, now, EndNoticeHousehold); err != nil {
t.Fatal(err)
}
if r, err := s.CustomerResidue("gone"); err != nil || r.OffsiteEndNotices != 1 || r.Total() < 1 {
t.Fatalf("the residue count does not see the notice record: %+v %v", r, err)
}
if err := s.PurgeCustomerResidue("gone"); err != nil {
t.Fatal(err)
}
if m, _ := s.EndNotices("gone"); len(m) != 0 {
t.Fatalf("the notice record outlived the household: %+v", m)
}
if err := s.SetEndNoticeForTest("ghost", "vikunja", "", now, now, EndNoticeHousehold); err != nil {
t.Fatal(err)
}
if n, err := s.PurgeOrphanRows(); err != nil || n["offsite_end_notices"] != 1 {
t.Fatalf("orphan sweep: %+v %v", n, err)
}
}
+1
View File
@@ -29,6 +29,7 @@ var orphanTables = []struct{ table, where string }{
{"offsite_abandon_requests", `customer_id NOT IN (SELECT customer_id FROM customer_configs) AND customer_id NOT IN (SELECT customer_id FROM hosts)`},
{"offsite_app_presence", `customer_id NOT IN (SELECT customer_id FROM customer_configs) AND customer_id NOT IN (SELECT customer_id FROM hosts)`},
{"offsite_large_requests", `customer_id NOT IN (SELECT customer_id FROM customer_configs) AND customer_id NOT IN (SELECT customer_id FROM hosts)`},
{"offsite_end_notices", `customer_id NOT IN (SELECT customer_id FROM customer_configs) AND customer_id NOT IN (SELECT customer_id FROM hosts)`},
}
// PurgeOrphanRows removes, in ONE transaction, the rows of boxes and households that no longer exist from the six
+4
View File
@@ -879,6 +879,10 @@ func (s *Store) migrate() error {
if err := s.migrateOffsiteEnded(); err != nil {
return err
}
// The notice lock (`09` §3 decision 315): the hub's own record that a household was told.
if err := s.migrateOffsiteEndNotices(); err != nil {
return err
}
// OS updates, guest fast lane (hub v0.130.0, `11` §8 step 2).
if err := s.migrateOSUpdates(); err != nil {
return fmt.Errorf("os_updates: %w", err)