hub v0.88.0 — the WAL that never was (R-172)

store.New opened the DB with `?_journal_mode=WAL&_busy_timeout=5000`, which is
mattn/go-sqlite3 syntax. The driver is modernc.org/sqlite, whose applyQueryParams
reads only _pragma/_time_format/_time_integer_format/_txlock/_inttotime and
IGNORES anything else WITHOUT AN ERROR. So the hub ran in rollback-journal mode
with busy_timeout=0 for its entire life while its own source said otherwise.

Surfaced as a false HOST STALE banner: in rollback-journal mode a reader excludes
a writer, so rendering an operator page blocks a host report; the hub 500s, the
agent waits its full 15-minute interval without retrying, and staleness fires at
30 minutes — two collisions is a false alarm plus an operator email. 13 collisions
in one pod lifetime; the alarm fired twice on 2026-08-02 for a host that was up
two days and reconciling throughout.

The observable that proved it: a 128 MB /data/hub.db with no -wal/-shm beside it
while the DB was open.

Fix: ?_pragma=journal_mode(WAL)&_pragma=busy_timeout(5000)&_txlock=immediate.
_txlock=immediate is not optional — database/sql's Begin() is DEFERRED, so a
read-then-write tx must upgrade its lock and a failed upgrade is
SQLITE_BUSY_SNAPSHOT, which busy_timeout does NOT retry; this store has 10+
db.Begin() sites and they are all write paths.

Every test asserts what the DATABASE reports, never the DSN string — a string
test would have passed for the whole life of the bug. Red-proof: restoring the
shipped DSN reproduces journal_mode="delete", the missing -wal, and the live
"database is locked (5) (SQLITE_BUSY)".

Operational consequence handled: a WAL DB cannot be copied by taking hub.db
alone — a bare `cat` opens cleanly and silently omits the newest writes. The
break-glass retrieval in operations/nodes.md used exactly that; it and the
recovery-inventory note are now WAL-aware.

hub/internal/store/store.go

@@ -51,9 +51,45 @@ type CustomerSummary struct {
 	DiskSummary     string
 }
 
+// sqliteDSNParams are the connection pragmas, and getting the SYNTAX right is the whole point.
+//
+// ── R-172: this DSN was WRONG for the hub's entire life, and it failed SILENTLY ──────────────────
+//
+// It used to read `?_journal_mode=WAL&_busy_timeout=5000`. That is **mattn/go-sqlite3** syntax. This
+// hub uses **modernc.org/sqlite**, whose `applyQueryParams` reads only `_pragma`, `_time_format`,
+// `_time_integer_format`, `_txlock` and `_inttotime` — anything else is **ignored without an error**.
+// So the hub ran in the default rollback-journal mode with busy_timeout=0 while its own source said
+// otherwise: a configuration asserting an invariant the code did not provide, the same class as the
+// comments in `CLAUDE.md`'s false-invariant table.
+//
+// The observable that proved it: a 128 MB `/data/hub.db` with **no `-wal`/`-shm` file beside it while
+// the database was open**. In WAL mode those files must exist. Consequence, measured on 2026-08-02:
+// 13 `SQLITE_BUSY` collisions in one pod lifetime, each returning HTTP 500 to a host report, and two
+// consecutive misses crossing the 30-minute staleness threshold — a false `host_stale` alarm plus an
+// operator e-mail for a host that was up and healthy throughout.
+//
+// Each parameter, and why it is not optional:
+//
+//   - journal_mode(WAL) — in rollback-journal mode a writer excludes readers and vice versa, so
+//     rendering an operator page could block a host report. WAL lets readers and one writer proceed
+//     concurrently. It is a property of the DATABASE FILE, so it persists once set.
+//   - busy_timeout(5000) — writers still serialise against each other. Without a timeout SQLite
+//     returns SQLITE_BUSY *immediately* rather than waiting; 5 s is far longer than any write here.
+//   - txlock=immediate — THE ONE THAT IS EASY TO MISS. `database/sql`'s Begin() is DEFERRED by
+//     default, so a transaction that reads and then writes must upgrade its lock, and a failed
+//     upgrade returns SQLITE_BUSY_SNAPSHOT, which **busy_timeout does not retry**. This store has
+//     10+ `db.Begin()` sites and they are all write paths (customer delete/reset, wg, appliance,
+//     pbsdr, telemetry, log bundles). Taking the write lock up front converts that un-retryable
+//     failure into an ordinary wait covered by busy_timeout above. WAL + busy_timeout WITHOUT this
+//     would leave a known un-retryable path open and ship half a fix.
+//
+// TestStorePragmasAreActuallyApplied asserts what the DATABASE reports, never what string was passed
+// — asserting the DSN would have passed happily for the entire life of the bug.
+const sqliteDSNParams = "?_pragma=journal_mode(WAL)&_pragma=busy_timeout(5000)&_txlock=immediate"
+
 // New creates a new store and initializes the schema.
 func New(dbPath string, logger *log.Logger) (*Store, error) {
-	db, err := sql.Open("sqlite", dbPath+"?_journal_mode=WAL&_busy_timeout=5000")
+	db, err := sql.Open("sqlite", dbPath+sqliteDSNParams)
 	if err != nil {
 		return nil, fmt.Errorf("opening database: %w", err)
 	}