From a1a6c73fe1329e5f92d497d43398f972cba39f30 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Sun, 6 Sep 2026 11:48:57 +0200 Subject: [PATCH] =?UTF-8?q?SPIKE:=20an=20upgrade=20test=20that=20runs=20ag?= =?UTF-8?q?ain=20=E2=80=94=20and=20a=20real=20defect=20in=20our=20own=20bo?= =?UTF-8?q?okstack=20template?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit R-449. Until today one app upgrade out of 53 had ever been measured, by hand, and the whole update arc was designed against that single data point. C3 first: the negative control, whose TO image exits immediately, came back failed. That is what makes the greens mean anything, and it cost 556s because a negative is only honest if it waits out the full settle window. Seven edges, three apps. All five real catalog upgrades kept the customer's data. The finding that changes an assumption the arc was carrying: whether an upgrade can be UNDONE is a property of the individual APP, not of upgrades. Docmost refuses - 'corrupted migrations: previously executed migration 20260213T085259-notifications is missing' - and privatebin does not. That reproduces the Nextcloud result on a second app by a DIFFERENT mechanism, so the struck word 'rollback' now rests on two measurements instead of one. The finding nobody was looking for, R-459: our own bookstack template moves MariaDB across a major and sets no MARIADB_* env at all, so the engine logs that the datadir upgrade it requires is being skipped, and serves anyway. The cause is assigned rather than guessed - the app half alone produces no upgrade line, both edges that move the engine produce it - which is exactly what decomposing E3 into E3a and E3b was for. It also explains why E3's abort looked like it worked: the datadir was never converted. Whether that ever breaks is NOT established, and the row says so. Also opened: R-460 (bookstack's file half cannot be seeded headlessly), R-461 (target-selection.md names a venue that does not exist and fences a VM that is gone), R-462 (the widening, costed with this run's real numbers - and the cost is dominated by fixtures, which do not amortise). Teardown all three layers, hub checked rather than asserted. local-lvm read 30.50 percent before and after. The capability map was deliberately NOT edited: this measured apps, not the product. --- REPORT.md | 198 +++++++++++--- STATUS.md | 48 +++- .../architecture/09-update-architecture.md | 59 +++- .../audits/SPIKE-upgrade-test-2026-09-06.md | 219 +++++++++++++++ .../evidence/C2/abort-states.json | 8 + .../evidence/C2/compose-full.log | 11 + .../evidence/C2/migration-lines.txt | 0 .../evidence/C2/run.log | 14 + .../evidence/C2/to-states.json | 8 + .../evidence/C2/verdict.json | 23 ++ .../evidence/C3/abort-states.json | 8 + .../evidence/C3/compose-full.log | 5 + .../evidence/C3/migration-lines.txt | 0 .../evidence/C3/run.log | 14 + .../evidence/C3/to-states.json | 8 + .../evidence/C3/verdict.json | 23 ++ .../evidence/E1/abort-states.json | 8 + .../evidence/E1/compose-full.log | 11 + .../evidence/E1/migration-lines.txt | 0 .../evidence/E1/run.log | 14 + .../evidence/E1/to-states.json | 8 + .../evidence/E1/verdict.json | 23 ++ .../evidence/E2/abort-refusal.txt | 91 +++++++ .../evidence/E2/abort-states.json | 20 ++ .../evidence/E2/compose-full.log | 256 ++++++++++++++++++ .../evidence/E2/migration-lines.txt | 6 + .../evidence/E2/run.log | 13 + .../evidence/E2/to-states.json | 20 ++ .../evidence/E2/verdict.json | 23 ++ .../evidence/E3/abort-states.json | 14 + .../evidence/E3/compose-final.log | 67 +++++ .../evidence/E3/compose-full.log | 67 +++++ .../evidence/E3/migration-lines.txt | 6 + .../evidence/E3/run.log | 14 + .../evidence/E3/to-full.log | 95 +++++++ .../evidence/E3/to-states.json | 14 + .../evidence/E3/verdict.json | 25 ++ .../evidence/E3a/abort-states.json | 14 + .../evidence/E3a/compose-final.log | 114 ++++++++ .../evidence/E3a/migration-lines.txt | 6 + .../evidence/E3a/run.log | 14 + .../evidence/E3a/to-full.log | 140 ++++++++++ .../evidence/E3a/to-states.json | 14 + .../evidence/E3a/verdict.json | 25 ++ .../evidence/E3b/abort-states.json | 14 + .../evidence/E3b/compose-final.log | 183 +++++++++++++ .../evidence/E3b/migration-lines.txt | 1 + .../evidence/E3b/run.log | 14 + .../evidence/E3b/to-full.log | 184 +++++++++++++ .../evidence/E3b/to-states.json | 14 + .../evidence/E3b/verdict.json | 25 ++ .../evidence/summary.json | 77 ++++++ documentation/backlog/CLOSED-ITEMS.md | 1 + documentation/backlog/OPEN-ITEMS.md | 5 +- documentation/backlog/ROADMAP.md | 2 +- 55 files changed, 2234 insertions(+), 54 deletions(-) create mode 100644 documentation/audits/SPIKE-upgrade-test-2026-09-06.md create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/C2/abort-states.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/C2/compose-full.log create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/C2/migration-lines.txt create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/C2/run.log create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/C2/to-states.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/C2/verdict.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/C3/abort-states.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/C3/compose-full.log create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/C3/migration-lines.txt create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/C3/run.log create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/C3/to-states.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/C3/verdict.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E1/abort-states.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E1/compose-full.log create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E1/migration-lines.txt create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E1/run.log create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E1/to-states.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E1/verdict.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E2/abort-refusal.txt create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E2/abort-states.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E2/compose-full.log create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E2/migration-lines.txt create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E2/run.log create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E2/to-states.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E2/verdict.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3/abort-states.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3/compose-final.log create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3/compose-full.log create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3/migration-lines.txt create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3/run.log create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3/to-full.log create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3/to-states.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3/verdict.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/abort-states.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/compose-final.log create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/migration-lines.txt create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/run.log create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/to-full.log create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/to-states.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/verdict.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/abort-states.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/compose-final.log create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/migration-lines.txt create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/run.log create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/to-full.log create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/to-states.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/verdict.json create mode 100644 documentation/audits/upgrade-spike-2026-09-06/evidence/summary.json diff --git a/REPORT.md b/REPORT.md index 5b93e8b7..b960bff1 100644 --- a/REPORT.md +++ b/REPORT.md @@ -1,62 +1,172 @@ -# REPORT — update arc slice 3: the version freeze (2026-09-06) +# REPORT — SPIKE: an upgrade test that runs again (2026-09-06) *Overwritten each session. Nothing durable lives only here.* -## What this session changed in THIS repo +> **C3 FIRST, because everything else is conditional on it.** The negative control — an edge whose TO +> image is `alpine:3.20`, which pulls cleanly and exits immediately — came back **`failed`** +> (`healthy_after: false`, `seed_read_after: false`, `seed_read_before: true`). **The harness can say +> no, so its greens mean something.** It also cost the most wall clock of any edge, 556 s, because a +> negative is only honest if it waits out the full settle window. -| file | change | +## 1. Confirmed baselines — none had moved + +| repo | task's baseline | found | +|---|---|---| +| app-catalog-felhom.eu | `7b9b9b34a5ee` | `7b9b9b34a5ee` | +| felhom.eu | `417df06f3529` | `417df06f3529` | +| felhom-controller | `bab82c4` (v0.235.0) | **not touched** | + +Highest `R-` id: **458**, confirmed. Minted **R-459 … R-462**. No version bump, no release, no golden. + +## 2. The verdict record — all seven edges + +Full JSON per edge in `documentation/audits/upgrade-spike-2026-09-06/evidence//verdict.json`. + +| edge | app | from → to | verdict | data after | **abort** | TO settle | total | +|---|---|---|---|---|---|---|---| +| **C3** | privatebin | `2.0.5` → **`alpine:3.20`** | **failed** | no | starts-and-serves | 421.1 s | 556.0 s | +| C2 | privatebin | `2.0.5` → `2.0.5` | proven | yes | starts-and-serves | 0.1 s | 6.4 s | +| **E1** | privatebin | `1.7.5` → `2.0.5` | **proven** | yes | **starts-and-serves** | 5.2 s | 21.5 s | +| **E2** | docmost | `0.25.3` → `0.95.0` | **proven** | yes | **REFUSES** | 10.7 s | 305.1 s | +| **E3** | bookstack | app `25.02.2`→`26.05.2` + mariadb `11.6`→`12.3` | **proven** | yes | starts-and-serves¹ | 15.7 s | 77.5 s | +| E3a | bookstack | app only, engine held | **proven** | yes | starts-and-serves | 15.7 s | 71.8 s | +| E3b | bookstack | engine only, app held | **proven** | yes | starts-and-serves¹ | 0.2 s | 48.8 s | + +¹ and §4 is why that is not the good news it looks like. + +**C1 passed on every edge, including C3.** A fixture that cannot prove itself first proves nothing +after. + +## 3. Quoted verbatim + +**E2's refusal — the abort of docmost:** + +``` +{"level":"error","context":"DatabaseMigrationService", + "msg":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing"} +{"level":"error","context":"DatabaseMigrationService","msg":"Failed to run database migration. Exiting program."} +``` + +**E2's migration, at the TO step** (six such lines, one shown): + +``` +{"level":"info","context":"DatabaseMigrationService","msg":"Migration \"20260213T085259-notifications\" executed successfully"} +``` + +**E3/E3b, at the moment MariaDB 12.3 first started on the 11.6 datadir:** + +``` +[Note] [Entrypoint]: MariaDB upgrade (mariadb-upgrade or creating healthcheck users) required, + but skipped due to $MARIADB_AUTO_UPGRADE setting +``` + +**E3a, the app half alone: no such line at all.** + +## 4. The two findings + +**(a) Whether an upgrade can be UNDONE is a property of the app, not of upgrades.** docmost refuses; +privatebin does not. This independently reproduces the Nextcloud finding on a second app **by a +different mechanism** — Nextcloud refused on a version comparison, docmost on its migration ledger. So +`09-update-architecture.md` §4's ruling now rests on two measurements, not one. **And the arc was +carrying an assumption that there is one answer for all 53 apps. There is not.** + +**(b) R-459 — a real defect in our own catalog, found by accident.** The bookstack template moves +MariaDB across a major and sets **no `MARIADB_*` env at all**, so the image skips the datadir upgrade +it says it requires. **The cause is assigned, not guessed:** E3a (app half, engine held) produces no +upgrade line; E3 and E3b (both move the engine) produce it. **A bundled edge could never have said +which half** — which is exactly what the decomposition existed for. **It also explains why E3's abort +"worked": the datadir was never converted, so 11.6 could still read it.** Whether that ever breaks is +**not established** and the row says so. + +## 5. What it cost — measured, for costing the widening + +| | | |---|---| -| `documentation/architecture/09-update-architecture.md` | **§3 gains the fourth dated operator ruling** (2026-09-06, Option 1); **§5 rewritten** from a proposed shape into the shipped mechanism — the pin, the stored definition, the render table, the four writers, the startup ordering, and §5.6 the trap it set for slice 2; slice 3 moved to shipped; two limitations added (§8.4 frozen-whole, §8.5 the `.felhom.yml` asymmetry). | -| `documentation/architecture/02-controller-module-map.md` | its *"copy compose + `.felhom.yml`"* line stopped being true and is corrected; §1/§2 of the app-definition seam now carry a banner saying they describe ≤ v0.234.0 and why they are kept. | -| `documentation/tests/VALIDATION-update-slice3-2026-09-06.md` | CREATED — the live evidence. | -| `documentation/backlog/OPEN-ITEMS.md` | **R-447, R-441, R-438 and R-455 closed and moved out**; **R-458 opened**. | -| `documentation/backlog/CLOSED-ITEMS.md` | the four closed rows, compressed, each naming `bc47dd4ef997` as the commit whose `git show` returns the original. | -| `documentation/backlog/ROADMAP.md` | the arc item collapsed to slices 1/1b/2/3 shipped; **R-448 named as the new head of the arc**. | -| `documentation/architecture/00-capability-map.md` | one new row, **PROVEN-LIVE**; the pre-v0.235.0 row relabelled rather than deleted, because every box below v0.235.0 still behaves that way. | -| `STATUS.md` | new lead + item 10; **items 4 and R-455 closed**. | +| successful edge | **6.4 – 305.1 s**, median **71.8 s** | +| failing edge | **556 s** — ~8× a positive | +| 7 edges total | ~18 min harness time + ~35 min build-out and two fixture iterations | +| disk, 3 apps / 11 images | **5.07 GB** images, 6.0 GB guest | +| naive extrapolation to 53 | ~90 GB, ~1 h harness time | -## The ruling, recorded +**The extrapolation understates it by an order of magnitude, and that is the finding.** Two of three +apps needed a bespoke seed route, one needed two attempts and a discarded approach, and one can only +ever be half-proven. **Fixture time scales with apps and does not amortise.** → **R-462**, which asks +the operator for scope rather than proposing one. -**2026-09-06, Option 1: freeze the version, keep the fixes flowing.** The document records *what the -ruling looked at*, not just its outcome: the old behaviour had two halves — a restart silently -changing an app's VERSION (unwanted), and template corrections plus self-healing reaching a deployed -app (worth keeping) — and the ruling keeps the second while removing the first. +## 6. Apps with no non-browser seed route -## Register — 203 open rows before, 203 after; closed 163 → 167 +**None was fully blocked; one is half-blocked.** privatebin and docmost have clean HTTP APIs. +**BookStack has neither an API token nor a usable HTTP login headlessly** — its template's `https` +`APP_URL` makes the session cookies `secure`, so curl over plain http gets **419 Page Expired** on +every login, which looks exactly like a wrong password. Its database half is provable through +`php artisan` (seed and readback are *different* commands, and the readback runs its own negative +control on every call). **Its FILE half cannot be seeded headlessly at all** → **R-460**. Nothing was +planted by hand anywhere. -Four closed and moved, one opened. The count is level by coincidence, not by inaction. +## 7. Evidence, copied off after EACH edge + +`felhom.eu/documentation/audits/upgrade-spike-2026-09-06/evidence/` — 48 files, 340 KB, pulled to +DooPlex after each edge and again at the end, before any teardown. Scanned for secrets before commit: +no token, no password, no generated key. The only `spike-*` strings are seeded account names from a +guest that no longer exists. + +## 8. Teardown — all three layers + +| layer | result | +|---|---| +| **1 — machine** | guest **9401 destroyed**; `pct list` shows only 9201. `scratch-upg` storage **removed**. Downloaded LXC template deleted. | +| **2 — host** | `local-lvm` **30.50 % before and after — never touched**, which was the whole point of siting the guest off it. `local` 19 595 164 → 19 605 084 KiB (+9.7 MB). `pct fstrim 9401` before destroy: **52.8 GiB trimmed**. | +| **3 — hub** | **Checked, not asserted:** `/hosts` lists exactly `demo-felhom-8363b5` and `demo-hp-bb76ea`; 0 customers created. **This run created no customer, no appliance and no host record.** | + +**No felhom-controller was in the path at any point.** Raw `docker compose` throughout. + +## 9. Register — 203 open before, 206 after; closed 167 → 168 | row | disposition | |---|---| -| **R-447** | **CLOSED** — slice 3 shipped, controller v0.235.0 | -| **R-441** | **CLOSED BY MEASUREMENT** — the restore now pins to what the unit captured; the render obeys it. The live half is §3/§6b of the validation file, not a code reading | -| **R-438** | **CLOSED** — both halves discharged: documented 2026-09-02, behaviour changed 2026-09-06 | -| **R-455** | **CLOSED** — the operator added a Docker Hub PAT | -| **R-458** | **OPENED** (P3-LOW, CC) — `.felhom.yml` keeps flowing to a frozen app, so it can receive a health check written for a newer version. **False alarm, never data loss.** The row states what would settle it by measurement rather than by code | +| **R-449** | **CLOSED** — harness built, run, and proven by a red negative control | +| **R-459** | OPENED, P2-MEDIUM — the skipped MariaDB datadir upgrade in our own bookstack template. *CC measures the consequence, VIKTOR rules on a fleet-wide env change* | +| **R-460** | OPENED, P3-LOW, CC — bookstack's file half is unprovable headlessly | +| **R-461** | OPENED, P3-LOW, CC — `target-selection.md` names a venue that does not exist and fences a fixture that is gone | +| **R-462** | OPENED, P2-MEDIUM — the widening, costed with real numbers. *VIKTOR rules on scope* | -## Live validation +## 10. The capability map was NOT edited, and that is deliberate -Full evidence: `documentation/tests/VALIDATION-update-slice3-2026-09-06.md`. **Two REAL catalog pushes -travelling the REAL 15-minute cycle**, both reverted in the same session; the catalog tree is -byte-identical to `8220f8d` afterwards. +**This run measured apps, not the product.** Nothing the platform can do changed: no controller code, +no version, no behaviour. Editing the map reflexively would record a capability the product did not +gain. Said here rather than left silent. -- **Scenario A** — a non-image change reached the pinned app (08:01:51Z), container untouched. -- **Scenario B** — an image change did not (08:20:29Z), and the restart afterwards took **0.1 s, did - not recreate the container, and never pulled the new image** — against **18.3 s with a pull** for the - identical sequence measured before the change. -- **Scenario D** — the Update button still moved the version, with the pin advancing **17 s before** - the pull completed. -- **Scenario G** — the frozen app read „Frissítés elérhető — 56 napja" while the other eight read - „Naprakész". -- **Teardown** — the container is back on the baseline digest `sha256:eaeea1e4…`, byte for byte. +## 11. Claims in the task that turned out to be wrong, named -**A real gap was found by the live run and fixed in it:** the stored definition did not follow the -fixes delivered after the pin, so the first freeze would have reverted them — silently undoing the -half of the ruling that says fixes keep flowing. Section 6 of the validation file keeps the -observation that proved it. +1. **§11's venue is stale, in two ways.** **`/mnt/nvme-1tb` does not exist** — the 1 TB NVMe is at + `/mnt/hdd_1`, the enrolled user-data drive, i.e. the same disk under a different path; the scratch + storage went at *its* root, honouring the rule's reason. And **`drill-r50` (VM 300) is gone** — + `qm list` returns nothing on demo-hp, so that fence protects nothing today. → **R-461**. +2. **§6's edges were all real and all resolvable.** Every one of the eleven images was verified against + its registry before use; none had to be substituted. The task asked to say so if any had. +3. **§8 expected bookstack to be "the one most likely to be hard" — correct, and for a reason the task + did not name.** The blocker was not the missing API token; it was that the template's `https` + `APP_URL` makes the session cookies `secure`, so no http login can ever work. Two independent + blockers, and only one was anticipated. +4. **§10.2's "gate on each command's own exit code" had to be broken once, deliberately and in the + open.** `bookstack:reset-mfa` exits **1 for a user it found and 1 for one it did not**, so the exit + code carries no information; the discriminator is the output, required positive with the not-found + sentence required absent. Stated in the fixture's docstring rather than done quietly. +5. **§9's verdict shape needed no change** and is now recorded in `09-update-architecture.md` §6 as the + contract Slice 6 carries. -## Sibling repos +## 12. Observations — noticed, documented, NOT acted on -- `felhom-controller` **v0.235.0** — `8a0e0a59adc7` + `2a56f557d048`, deployed to demo-hp, healthy. -- `app-catalog-felhom.eu` — `dc7e548`, `09b4ff5` (live-test) and `1798ce6`, `17cc784` (reverts), plus - `7b9b9b3` recording them as a measurement rather than a release. +1. **`docker compose logs` only shows containers that currently exist**, so the abort erases the TO + step's output from any later capture — **the single most important line of this run survived only + because it had already been extracted.** Fixed mid-run (`to-full.log` is now written at the TO + step) and E3 re-run to get clean evidence. **FILED: R-449's closure records it; the harness change + is committed.** **NOT-A-FINDING as a separate row: it is a harness bug that was found and fixed + inside the same session, with the fix committed and the affected edge re-measured — there is no + residue for a row to track.** +2. **A negative edge costs ~8× a positive.** **NOT-A-FINDING: it is a measured cost recorded in R-462, + which is where the widening will be scheduled from; a second row would duplicate it.** +3. **`bookstack:reset-mfa`'s help says `[options]` but a positional argument is rejected with "No + arguments expected"** in 25.02.2 — correct behaviour that reads as a missing feature. + **NOT-A-FINDING: it is upstream's interface, not ours, and it costs us nothing now that the fixture + documents it.** diff --git a/STATUS.md b/STATUS.md index c45a9ac6..76d217ee 100644 --- a/STATUS.md +++ b/STATUS.md @@ -1,9 +1,14 @@ # STATUS — what works, what's broken, what's next -**Updated 2026-09-06 — a restart no longer changes which version an app runs. Fixes still arrive +**Updated 2026-09-06 (second pass) — I built a machine that upgrades a real app with real data in it +and then asks the app whether the data is still there. Three apps, five real upgrades: the data +survived every time. It also found a genuine problem in our own BookStack setup. ONE NEW THING NEEDS +YOU: item 11 — how wide should I take this?** + +**Earlier 2026-09-06 — a restart no longer changes which version an app runs. Fixes still arrive every 15 minutes, and a broken app definition still repairs itself. Only the Update button moves a -version now. Live on the HP (0.235.0). NOTHING IS WAITING ON YOU — and two old items are now closed: -the Hetzner e-mails are answered, and the Docker Hub login is in place.** +version now. Live on the HP (0.235.0). Two old items closed: the Hetzner e-mails are answered, and +the Docker Hub login is in place.** **Earlier 2026-09-03 — you spotted that OpenGist had no label. You were right, and it was a real gap: the label only appeared on apps something had restarted. Fixed and live (0.234.0). Every app on both @@ -31,7 +36,7 @@ not an evening's work.** *This section is allowed to be longer than one screen, and each item says what happens if you do nothing.* -1. **Nothing is waiting on you.** Item 4 (the Hetzner e-mails) is answered and is being handled in a separate session. Item 7 — the safety-copy decision — is the one open question, and it is **not urgent any more**: the thing that made it urgent was that a restart could upgrade an app behind your back, and as of today it cannot. Item 10 is new and needs nothing from you. Item 5's alarm mail can now be ignored for good. Otherwise: Both problems the overnight test found are fixed and proven on +1. **One thing is waiting on you: item 11 (how wide to take the upgrade testing).** Item 4 (the Hetzner e-mails) is answered and is being handled in a separate session. Item 7 — the safety-copy decision — is the one open question, and it is **not urgent any more**: the thing that made it urgent was that a restart could upgrade an app behind your back, and as of today it cannot. Item 10 is new and needs nothing from you. Item 5's alarm mail can now be ignored for good. Otherwise: Both problems the overnight test found are fixed and proven on the real machines: - the background job that could delete a live restore's lock now waits its turn — and the check that finds the next one like it is a test, not a comment, so it cannot come back quietly; @@ -191,6 +196,41 @@ nothing.* version and look unwell when it is fine. **It cannot lose data — the worst case is a false alarm.** Freezing that file too would break the „Frissítés elérhető" label, which is a worse trade. +11. **How wide should I take the upgrade testing? This is the one decision from today, and it is + about money and time, not about safety.** + + **What I built.** A machine that installs an app, puts real data in through the app's own front + door, upgrades it, and then asks the app for the data back. Not "did it start" — the box has + already fooled us that way once. + + **I also taught it to fail.** Before believing anything, I pointed it at an upgrade I knew was + broken. It came back red. That is why I trust the greens. + + **What it found, on three apps and five real upgrades.** + - **The data survived every single time.** That is the good news and it is worth having. + - **Whether an upgrade can be UNDONE depends on the app, not on upgrades.** Docmost will not go + back — the old version refuses to start on the changed data. PrivateBin goes back fine. **We + had been assuming one answer for all 53 apps. There isn't one.** + - **And it found a real problem in our own BookStack setup.** Our template moves the database + engine to a new major version, and the engine says, in its own words, that the conversion it + needs is being *skipped*. It works today. **I did not measure whether it ever breaks**, and I + am not going to guess. + + **The decision: how many of the 53 apps do I test?** + + - **Only the apps that keep data in a database (~25).** Today's run showed the undo question only + ever bites there. **Cost: roughly a day of my time.** + - **All 53.** Complete, and it gives us a list nobody has. **Cost: several days**, and the reason + is not the machines — it is that **each app needs its own hand-written way in**, and that work + does not get cheaper the more you do. Two of today's three needed one, and one of them took two + attempts. + + **My pick: the database ones first.** It answers the question that changes the product, and if it + goes well the rest is a decision you can take later with better numbers. + + **If you do nothing:** nothing breaks. The machine is built and committed, so it does not go + stale, and the BookStack problem is written down and waiting either way. + 8. **`demo-hp`'s network setup does not match our own notes** (R-338) — the machine works, the page is wrong, or the other way round. **If you do nothing:** the page keeps misleading the next session, as it misled one by an hour. diff --git a/documentation/architecture/09-update-architecture.md b/documentation/architecture/09-update-architecture.md index 08799842..f4daec9e 100644 --- a/documentation/architecture/09-update-architecture.md +++ b/documentation/architecture/09-update-architecture.md @@ -144,8 +144,31 @@ with a positive control proving the data is intact, only unreachable by the old | **ABORT** | before anything migrated | stop, put the old image back, the app runs again | | **RESTORE FROM A COPY** | after a migration ran | the data restore is the whole remedy | -There is no third. And per §5 below, a restore's image-level undo currently has a ≤15-minute -half-life because the syncer overwrites it (**R-441**). +There is no third. + +### 4.1 MEASURED 2026-09-06 — and the abort turns out to be a property of the APP, not of upgrades + +`SPIKE-upgrade-test-2026-09-06.md` upgraded three real apps with real data in them and then attempted +the abort on each. **All five real catalog upgrades kept the customer's data.** The abort did not +behave the same way twice: + +| app | abort | why | +|---|---|---| +| **docmost** `0.25.3`→`0.95.0` | **REFUSES** | the old code finds migration ledger entries it does not know: *"corrupted migrations: previously executed migration 20260213T085259-notifications is missing"*, then *"Failed to run database migration. Exiting program."* | +| **privatebin** `1.7.5`→`2.0.5` | **works** | file-backed, no database, no schema — a major version moves no data | +| **bookstack** app+engine | **works, misleadingly** | only because the MariaDB datadir upgrade was skipped and never happened — see R-459 | + +**This puts TWO independent measurements behind the ruling above, by two unrelated mechanisms:** +Nextcloud refused on an explicit version comparison; docmost refuses on its migration ledger. The word +"rollback" was already struck; it is now struck on evidence rather than on one case. + +**And it adds a distinction this document did not have: there is no single answer to "can this update +be undone". There are apps where it can and apps where it cannot, and the only way to know which is to +MEASURE THAT APP.** Any design that assumes one answer for all 53 is designing against a fact that was +checked and is false. + +Per §5 below, a restore's image-level undo used to have a ≤15-minute half-life because the syncer +overwrote it (**R-441**) — closed in v0.235.0. --- @@ -250,10 +273,32 @@ earlier feature is the failure mode to look for whenever a file changes meaning. | **2** | **One badge says whether the app is current** — „Naprakész" / „Frissítés elérhető — N napja", from `catalog_since`. No version number. | **SHIPPED, controller v0.233.0 + catalog `69761cf` (2026-09-02)** | | **3** | **The compose file becomes DERIVED** — the pin in `app.yaml` wins; the syncer renders instead of copying. | **SHIPPED, controller v0.235.0 (2026-09-06)** — operator ruling §3.4 | | **4** | **A guarded update** — verified-backup precondition, abort-on-failure, and the truth at the moment of action rather than 5m16s later (R-443). | OPEN — R-448 | -| **5** | **An upgrade test** — prove a real one-major upgrade end to end, including the abort path. | OPEN — R-449 | +| **5** | **An upgrade test that runs again** — a harness that upgrades a real app with real data in it and asks the app for the data back. | **SHIPPED, `app-catalog/scripts/upgrade-test.py` (2026-09-06)** — 7 edges, 3 apps; see §4.1 and §10 | | **6** | **A version sequence** — updates automatic within a major, a human across one; **an engine change gets its own edge.** | OPEN — R-450 | | **7** | **A fleet sweep pipeline** — the operator can see, and move, how far behind every box is. | OPEN — R-451 | +### The verdict record — the contract Slice 6 carries + +Decided here rather than invented twice. The harness writes one of these per edge, beside its +evidence; Slice 6 puts the same shape in the catalog. + +```json +{"harness_version": 1, "app": "bookstack", + "from": {"bookstack": "…:25.02.2", "bookstack-db": "mariadb:11.6"}, + "to": {"bookstack": "…:26.05.2", "bookstack-db": "mariadb:12.3"}, + "verdict": "proven | failed | inconclusive", + "seed_read_before": true, "seed_read_after": true, "healthy_after": true, + "migration_observed": "verbatim log line, or null", + "abort": "starts-and-serves | refuses | starts-data-gone | not-attempted", + "abort_detail": "the refusal quoted verbatim, or null", + "duration_s": 0, "measured_at": "RFC3339", "evidence": "relative path"} +``` + +**`inconclusive` is a first-class verdict and must never be collapsed into `failed`.** "We could not +measure it" and "it does not work" are different facts, and only one of them is about the app. +**`migration_observed` is a quoted line, never an inference from timing** — the value of both the +Nextcloud and the docmost findings was the exact sentence the app printed. + **The rule slice 6 inherits, recorded now while it is cheap:** an engine change gets its own edge, never bundled with an app version bump. `bookstack` moved the application *and* MariaDB 11.6 → 12.3 in one commit (`0b73e5e`); that is two migrations behind one edge, and an unreadable failure when it @@ -353,7 +398,13 @@ Version strings stay in the logs, the API and the hub. 6. **The Update button is still unguarded.** It takes no backup, has no rollback, and can still attempt a multi-major jump the app will refuse (R-40). **Slice 3 did not change that and must not be read as having done so** — the precondition is slice 4 (R-448). -7. **The hub does not record image tags at all.** Its report's container payload carries name, state, +7. **An engine major can be applied without its datadir upgrade, and nothing notices.** Measured + 2026-09-06: the catalog's own bookstack transition starts MariaDB 12.3 on an 11.6 datadir, and the + image logs that the required upgrade was **skipped** because the template sets no + `MARIADB_AUTO_UPGRADE`. The app serves. Whether that ever breaks is **not** established. **R-459.** +8. **Only three of 53 apps have ever had an upgrade measured**, and one of them (bookstack) can only + be half-proven headlessly (**R-460**). The widening is **R-462**, costed with real numbers. +9. **The hub does not record image tags at all.** Its report's container payload carries name, state, CPU and memory, and no image field (spike §5). So the fleet view of §6 slice 7 needs a hub-side change; it is not derivable from what is already reported. diff --git a/documentation/audits/SPIKE-upgrade-test-2026-09-06.md b/documentation/audits/SPIKE-upgrade-test-2026-09-06.md new file mode 100644 index 00000000..42b28fe3 --- /dev/null +++ b/documentation/audits/SPIKE-upgrade-test-2026-09-06.md @@ -0,0 +1,219 @@ +# SPIKE — does a real app upgrade keep the customer's data, and can it be undone? (2026-09-06) + +> **THE ANSWER, IN ONE SENTENCE: on all five real catalog upgrades measured, the customer's data +> survived — and whether the upgrade can be UNDONE is not a property of upgrades at all, it is a +> property of the individual app, which is why it has to be measured per app rather than reasoned +> about.** +> +> **AND THE FINDING NOBODY WAS LOOKING FOR, which is a defect in our own catalog:** this repo's +> bookstack template moves MariaDB **11.6 → 12.3**, and MariaDB 12.3 comes up, says in its own words +> that a datadir upgrade is **required**, **skips it**, and serves anyway — because the template sets +> no `MARIADB_AUTO_UPGRADE`. The app works. The engine is running on a datadir it itself calls +> un-upgraded. **R-459.** + +**Class: spike.** No controller code, no product change, no customer's box. Everything ran inside a +throwaway LXC destroyed at the end (§7). **R-449** asked for "an upgrade test that runs again"; the +harness is `app-catalog-felhom.eu/scripts/upgrade-test.py` and it is now a maintained script, not an +audit artifact. + +--- + +## 1. C3 first — the harness can say no + +**Everything below is conditional on this, so it is reported first.** C3 is an edge whose TO image is +`alpine:3.20` — a real image that pulls cleanly and exits immediately. + +``` +verdict: failed seed_read_before: true seed_read_after: false healthy_after: false +TO settled=False in 421.1s :: {"privatebin": {"status": "restarting", "health": "unhealthy"}} +``` + +**C3 came back RED.** A harness that cannot fail a known-broken upgrade proves nothing with its +greens. It also cost the most wall-clock of any edge (556 s), because a negative is only honest if it +waits out the full settle window. + +**C1 (the seed reads back BEFORE the upgrade) passed on every edge**, including C3 — a fixture that +cannot prove itself first proves nothing after. + +## 2. The verdict table + +| edge | app | from → to | verdict | data after | **abort** | TO settle | total | +|---|---|---|---|---|---|---|---| +| **C3** | privatebin | `2.0.5` → **`alpine:3.20`** | **failed** | no | starts-and-serves | 421.1 s | 556.0 s | +| **C2** | privatebin | `2.0.5` → `2.0.5` (no-op) | proven | yes | starts-and-serves | 0.1 s | 6.4 s | +| **E1** | privatebin | `1.7.5` → `2.0.5` (major) | **proven** | yes | **starts-and-serves** | 5.2 s | 21.5 s | +| **E2** | docmost | `0.25.3` → `0.95.0` | **proven** | yes | **REFUSES** | 10.7 s | 305.1 s | +| **E3** | bookstack | app `25.02.2`→`26.05.2` **+** mariadb `11.6`→`12.3` | **proven** | yes | starts-and-serves¹ | 15.7 s | 77.5 s | +| **E3a** | bookstack | app only, engine held at `11.6` | **proven** | yes | starts-and-serves | 15.7 s | 71.8 s | +| **E3b** | bookstack | engine only `11.6`→`12.3`, app held | **proven** | yes | starts-and-serves¹ | 0.2 s | 48.8 s | + +¹ **and §4 is why that is not the good news it looks like.** + +## 3. The abort is a property of the APP, not of upgrades + +**E2, docmost — the abort REFUSES.** The old image will not start on the migrated database. Verbatim, +from the app's own log: + +``` +{"level":"error","context":"DatabaseMigrationService", + "msg":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing"} +{"level":"error","context":"DatabaseMigrationService","msg":"Failed to run database migration. Exiting program."} +``` + +The container then crash-loops: `{"docmost": {"status": "restarting", "health": "unhealthy", "exit": 1}}` +while `docmost-postgres` and `docmost-redis` stay healthy beside it. + +**This independently reproduces the Nextcloud finding on a second app — and by a DIFFERENT +mechanism.** Nextcloud refused on an explicit version comparison; docmost refuses because its +migration ledger contains entries the older code does not know about. Two apps, two unrelated causes, +same outcome. **`09-update-architecture.md` §4's ruling that "rollback" is the wrong word is now +supported by two independent measurements rather than one.** + +**E1, privatebin — the abort WORKS**, and the reason is structural: privatebin is file-backed with no +database and no schema, so a major version change moves no data. Its migration-line capture is empty, +which is a true negative rather than a missed observation. + +**So the vocabulary needs one more distinction than the arc currently has.** There is no single answer +to "can an update be undone". There are apps where it can and apps where it cannot, and **the only way +to know which is to measure that app**. This is the finding R-449 existed to produce. + +## 4. The finding nobody was looking for — R-459 + +E3 is the catalog's own bookstack transition, and it moves the **database engine** across a major at +the same time as the app. It came back `proven`, and its abort came back `starts-and-serves`. **Both +are true and both are misleading, and the decomposition is what showed it.** + +Verbatim, at the moment MariaDB 12.3 first started on the 11.6 datadir: + +``` +[Note] [Entrypoint]: MariaDB upgrade (mariadb-upgrade or creating healthcheck users) required, + but skipped due to $MARIADB_AUTO_UPGRADE setting +``` + +Confirmed by reading which engine actually served in each edge — not inferred: + +| edge | engine that served the TO step | upgrade line | +|---|---|---| +| E3 (app **and** engine) | `12.3.3-MariaDB-ubu2404` | **required, but skipped** | +| **E3a (app only)** | `11.6.2-MariaDB-ubu2404` | **none** | +| E3b (**engine only**) | `12.3.3-MariaDB-ubu2404` | **required, but skipped** | + +**E3a is the control that assigns the cause.** The app half alone produces no engine upgrade line at +all; both edges that move the engine produce it. **This is exactly what the decomposition was for, and +a bundled edge could never have said it.** + +`templates/bookstack/docker-compose.yml` sets **no `MARIADB_*` environment at all**, so +`MARIADB_AUTO_UPGRADE` is unset and the image's entrypoint declines to run `mariadb-upgrade`. + +**And this is why the abort "worked":** the datadir was never converted, so MariaDB 11.6 could still +read it — on the way back it says `MariaDB upgrade not required`. **The reversibility of E3 is a +side-effect of an upgrade that did not fully happen.** + +**What is NOT established, and the row says so:** whether running 12.3 on an unconverted 11.6 datadir +ever actually breaks. It did not break here. MariaDB itself calls the upgrade required; we measured +that it is skipped, and we did **not** measure a consequence. **Naming the gap is the finding; the +consequence is a separate measurement.** + +## 5. What each edge cost — the numbers the widening must be costed with + +**Measured, not estimated**, which is the whole point of §15.5 of the task. + +| | | +|---|---| +| wall-clock, successful edges | **6.4 s – 305.1 s**, median **71.8 s** | +| wall-clock, the failing edge | **556 s** — a negative costs ~8× a positive, because it must wait out the full settle window | +| total for 7 edges | **~18 minutes** of harness time, plus ~35 minutes of build-out and two fixture iterations | +| disk, 3 apps / 11 images | **5.07 GB** of images, 6.0 GB guest total | +| **naive extrapolation to 53 apps** | **~90 GB of images** and, at the median, ~1 h of harness time for one edge each — **but see the caveat below, which is the real cost** | + +**The real cost is not the machine, it is the fixture.** Two of the three apps needed a bespoke +non-browser seed route, and one of those (bookstack) needed two attempts and a discarded approach +(§6). **Harness time scales with apps; fixture time scales with apps too, and it does not amortise.** +Costing the widening from the 71.8 s median alone would understate it by an order of magnitude. + +## 6. Fixtures — what worked, what did not, and what cannot be done at all + +Every seed went in through the app's **own** interface. Nothing was written into a volume or a +database by hand (R-156). + +| app | seed route | readback route | notes | +|---|---|---|---| +| privatebin | its JSON paste API (HTTP POST) | HTTP GET of the same paste | **file-backed, no database — this single seed IS the file half**; there is no database half to seed | +| docmost | `POST /api/auth/setup` | `POST /api/auth/login` as the seeded user | the readback deliberately uses the app's own front door, and is version-stable across the 0.25→0.95 API churn | +| bookstack | `php artisan bookstack:create-admin` | `php artisan bookstack:reset-mfa --email=…` | **database half only — see below** | + +**Two dead ends, recorded because they cost time and will cost the next person the same:** + +1. **PrivateBin refuses a malformed paste envelope** with `{"status":1,"message":"Invalid data."}`. + `ct`, the IV and the salt must all be real base64. The first fixture attempt used placeholder + strings and was rejected — which read like an app failure and was a harness bug. +2. **BookStack cannot be verified over HTTP without TLS.** Its `APP_URL` comes from the template as + `https://${SUBDOMAIN}.${DOMAIN}`, so it marks its session and XSRF cookies **`secure`**; curl over + plain http stores neither, sends neither, and **every login POST returns 419 Page Expired** — which + looks exactly like a wrong password. The container serves no TLS. There is no http route to a + logged-in session without changing the app's own configuration, which would be measuring a + different app. + +**So bookstack's readback is `artisan` on both sides**, and two things were done to keep that honest: +the readback command is a *different* command from the seed and must FIND the record the other one +created; and **the fixture runs its own negative control on every call** — it also looks up an email +that cannot exist and requires the answer `A user where email=… could not be found.` A readback that +had broken into always saying "found" therefore fails instead of passing everything. + +**One thing genuinely cannot be done headlessly, and it is a gap, not a defect: the FILE half of +bookstack.** Seeding an uploaded image or attachment needs an API token that BookStack only mints +through a browser. **So for bookstack this run proves the database survived and says nothing about +uploaded files.** **R-460.** + +**A note on the standing "gate on the exit code" rule, because this run had to break it once.** +`bookstack:reset-mfa` asks for interactive confirmation, finds no TTY, and exits **1 in both cases** — +for a user it found and one it did not. The exit code carries no information there, so the +discriminator is the output, required positive AND with the not-found sentence required absent. Stated +rather than done quietly. + +## 7. Venue and teardown — all three layers + +Throwaway **LXC 9401 `upgrade-spike`** on `demo-hp`, Debian 13, 4 cores, 6 GB, created for this run. + +**Two things the runbook says about this venue are now out of date, and were worked around rather than +followed blindly:** + +- **`/mnt/nvme-1tb` does not exist.** The 1 TB NVMe is mounted at **`/mnt/hdd_1`** and is the enrolled + user-data drive. A dir storage `scratch-upg` was created **at its root** (the runbook's own rule — + a subdirectory fails the agent's `exactMount` check) and removed at teardown. The guest's disk was + deliberately kept **off `local-lvm`**, whose over-subscription is the runbook's real warning. +- **`drill-r50` (VM 300) no longer exists.** `qm list` returns nothing on this host. The runbook's + "do not destroy it" fence currently protects nothing. **R-461.** + +| layer | before | after | +|---|---|---| +| **1 — the machine** | guest 9401 created | **destroyed**; `pct list` shows only 9201. `scratch-upg` storage **removed**. Downloaded LXC template deleted. | +| **2 — the host** | `local-lvm` **30.50 %**, `local` 19 595 164 KiB | `local-lvm` **30.50 % — unchanged, never touched**; `local` 19 605 084 KiB (+9.7 MB). `pct fstrim 9401` before destroy: **52.8 GiB trimmed** | +| **3 — the hub** | 2 enrolled hosts | **2 enrolled hosts, 0 customers created.** Checked, not assumed: `/hosts` lists exactly `demo-felhom-8363b5` and `demo-hp-bb76ea`. **This run created no customer, no appliance and no host record.** | + +**No felhom-controller was in the path at any point** — raw `docker compose` throughout. The property +under test belongs to the app and its images; putting the controller in the path would have confounded +the two, which is the choice the persistence sweep made and stated. + +## 8. What was NOT measured + +- **50 of 53 apps.** Three is the sample that decides whether the idea is sound, not a fleet survey. +- **Whether an unconverted MariaDB datadir ever breaks** (§4). Measured that the upgrade is skipped; + did not measure a consequence. +- **The FILE half of bookstack and docmost.** privatebin's seed is a file; the other two prove the + database only. +- **Any edge with realistic data VOLUME.** Every seed here is one record. An upgrade that migrates + 10 GB may behave differently, and the durations above are therefore floors, not estimates. +- **Anything through felhom-controller**, deliberately (§7). + +## 9. Observations — noticed, documented, not acted on + +1. **`docker compose logs` only shows the containers that currently exist**, so the abort — which + replaces them — erases the TO step's output from any capture taken afterwards. **The single most + important line of this run survived only because it had already been extracted.** The harness now + writes `to-full.log` at the TO step. Same class as R-320, one layer down. +2. **A negative edge costs ~8× a positive** because it must wait out the settle window. If the + widening runs many edges, that asymmetry dominates the schedule, not the average. +3. **`bookstack:reset-mfa`'s help output disagrees with its argument parsing** in 25.02.2 — the usage + line says `[options]` and a positional argument is rejected with *"No arguments expected"*, which is + correct but reads as a missing feature. Upstream's problem, recorded because it cost a minute. diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/C2/abort-states.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/C2/abort-states.json new file mode 100644 index 00000000..f7a71c94 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/C2/abort-states.json @@ -0,0 +1,8 @@ +{ + "privatebin": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + } +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/C2/compose-full.log b/documentation/audits/upgrade-spike-2026-09-06/evidence/C2/compose-full.log new file mode 100644 index 00000000..7f23c17c --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/C2/compose-full.log @@ -0,0 +1,11 @@ +privatebin | [06-Sep-2026 11:25:20] NOTICE: fpm is running, pid 11 +privatebin | [06-Sep-2026 11:25:20] NOTICE: ready to handle connections +privatebin | 127.0.0.1 - - [06/Sep/2026:11:25:25 +0200] "GET / HTTP/1.1" 200 11465 "-" "Wget" "-" +privatebin | 172.18.0.1 - - [06/Sep/2026:11:25:25 +0200] "GET / HTTP/1.1" 200 22914 "-" "curl/8.14.1" "-" +privatebin | 172.18.0.1 - - [06/Sep/2026:11:25:25 +0200] "POST / HTTP/1.1" 200 156 "-" "curl/8.14.1" "-" +privatebin | 172.18.0.1 - - [06/Sep/2026:11:25:25 +0200] "GET / HTTP/1.1" 200 22914 "-" "curl/8.14.1" "-" +privatebin | 172.18.0.1 - - [06/Sep/2026:11:25:25 +0200] "GET /?pasteid=ee0b2b1da99ffa3d HTTP/1.1" 200 310 "-" "curl/8.14.1" "-" +privatebin | 172.18.0.1 - - [06/Sep/2026:11:25:25 +0200] "GET / HTTP/1.1" 200 22914 "-" "curl/8.14.1" "-" +privatebin | 172.18.0.1 - - [06/Sep/2026:11:25:25 +0200] "GET /?pasteid=ee0b2b1da99ffa3d HTTP/1.1" 200 310 "-" "curl/8.14.1" "-" +privatebin | 172.18.0.1 - - [06/Sep/2026:11:25:26 +0200] "GET / HTTP/1.1" 200 22914 "-" "curl/8.14.1" "-" +privatebin | 172.18.0.1 - - [06/Sep/2026:11:25:26 +0200] "GET /?pasteid=ee0b2b1da99ffa3d HTTP/1.1" 200 310 "-" "curl/8.14.1" "-" diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/C2/migration-lines.txt b/documentation/audits/upgrade-spike-2026-09-06/evidence/C2/migration-lines.txt new file mode 100644 index 00000000..e69de29b diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/C2/run.log b/documentation/audits/upgrade-spike-2026-09-06/evidence/C2/run.log new file mode 100644 index 00000000..258b4b41 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/C2/run.log @@ -0,0 +1,14 @@ +[09:25:19] C2: deploying privatebin at FROM {'privatebin': 'privatebin/pdo:2.0.5'} +[09:25:25] FROM settled=True in 5.2s :: {"privatebin": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}} +[09:25:25] privatebin: seeded paste id=ee0b2b1da99ffa3d +[09:25:25] privatebin: readback http=200 marker_present=True +[09:25:25] C1 (seed reads back BEFORE): True +[09:25:25] C2: swapping to TO {'privatebin': 'privatebin/pdo:2.0.5'} +[09:25:25] TO up -d rc=0 +[09:25:25] TO settled=True in 0.1s :: {"privatebin": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}} +[09:25:25] migration lines observed: 0 +[09:25:25] privatebin: readback http=200 marker_present=True +[09:25:25] RESULT (seed reads back AFTER): True +[09:25:25] C2: ABORT — putting the FROM images back +[09:25:26] privatebin: readback http=200 marker_present=True +[09:25:26] ABORT: app came back in 0.1s; data present=True \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/C2/to-states.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/C2/to-states.json new file mode 100644 index 00000000..f7a71c94 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/C2/to-states.json @@ -0,0 +1,8 @@ +{ + "privatebin": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + } +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/C2/verdict.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/C2/verdict.json new file mode 100644 index 00000000..14325c13 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/C2/verdict.json @@ -0,0 +1,23 @@ +{ + "harness_version": 1, + "edge": "C2", + "app": "privatebin", + "note": "no-op control: a version to ITSELF", + "from": { + "privatebin": "privatebin/pdo:2.0.5" + }, + "to": { + "privatebin": "privatebin/pdo:2.0.5" + }, + "verdict": "proven", + "seed_read_before": true, + "seed_read_after": true, + "healthy_after": true, + "migration_observed": null, + "abort": "starts-and-serves", + "abort_detail": null, + "duration_s": 0.1, + "measured_at": "2026-09-06T09:25:26Z", + "evidence": "evidence/C2", + "total_s": 6.4 +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/C3/abort-states.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/C3/abort-states.json new file mode 100644 index 00000000..f7a71c94 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/C3/abort-states.json @@ -0,0 +1,8 @@ +{ + "privatebin": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + } +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/C3/compose-full.log b/documentation/audits/upgrade-spike-2026-09-06/evidence/C3/compose-full.log new file mode 100644 index 00000000..fc1c9f9b --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/C3/compose-full.log @@ -0,0 +1,5 @@ +privatebin | [06-Sep-2026 11:24:43] NOTICE: fpm is running, pid 11 +privatebin | [06-Sep-2026 11:24:43] NOTICE: ready to handle connections +privatebin | 127.0.0.1 - - [06/Sep/2026:11:24:48 +0200] "GET / HTTP/1.1" 200 19657 "-" "Wget" "-" +privatebin | 172.18.0.1 - - [06/Sep/2026:11:24:48 +0200] "GET / HTTP/1.1" 200 22914 "-" "curl/8.14.1" "-" +privatebin | 172.18.0.1 - - [06/Sep/2026:11:24:48 +0200] "GET /?pasteid=d5cb8959c9012690 HTTP/1.1" 200 310 "-" "curl/8.14.1" "-" diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/C3/migration-lines.txt b/documentation/audits/upgrade-spike-2026-09-06/evidence/C3/migration-lines.txt new file mode 100644 index 00000000..e69de29b diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/C3/run.log b/documentation/audits/upgrade-spike-2026-09-06/evidence/C3/run.log new file mode 100644 index 00000000..1f771063 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/C3/run.log @@ -0,0 +1,14 @@ +[09:15:32] C3: deploying privatebin at FROM {'privatebin': 'privatebin/pdo:2.0.5'} +[09:15:38] FROM settled=True in 5.2s :: {"privatebin": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}} +[09:15:38] privatebin: seeded paste id=d5cb8959c9012690 +[09:15:38] privatebin: readback http=200 marker_present=True +[09:15:38] C1 (seed reads back BEFORE): True +[09:15:38] C3: swapping to TO {'privatebin': 'alpine:3.20'} +[09:15:41] TO up -d rc=0 +[09:22:42] TO settled=False in 421.1s :: {"privatebin": {"status": "restarting", "health": "unhealthy", "restarts": 0, "exit": 0}} +[09:22:42] migration lines observed: 0 +[09:24:42] app never answered on http://invalid:8080/ (last rc=6 code=000) +[09:24:42] RESULT (seed reads back AFTER): False +[09:24:42] C3: ABORT — putting the FROM images back +[09:24:48] privatebin: readback http=200 marker_present=True +[09:24:48] ABORT: app came back in 5.3s; data present=True \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/C3/to-states.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/C3/to-states.json new file mode 100644 index 00000000..04f4a178 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/C3/to-states.json @@ -0,0 +1,8 @@ +{ + "privatebin": { + "status": "restarting", + "health": "unhealthy", + "restarts": 0, + "exit": 0 + } +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/C3/verdict.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/C3/verdict.json new file mode 100644 index 00000000..795b0744 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/C3/verdict.json @@ -0,0 +1,23 @@ +{ + "harness_version": 1, + "edge": "C3", + "app": "privatebin", + "note": "NEGATIVE control: the TO image starts and exits immediately", + "from": { + "privatebin": "privatebin/pdo:2.0.5" + }, + "to": { + "privatebin": "alpine:3.20" + }, + "verdict": "failed", + "seed_read_before": true, + "seed_read_after": false, + "healthy_after": false, + "migration_observed": null, + "abort": "starts-and-serves", + "abort_detail": null, + "duration_s": 421.1, + "measured_at": "2026-09-06T09:24:48Z", + "evidence": "evidence/C3", + "total_s": 556.0 +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E1/abort-states.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/E1/abort-states.json new file mode 100644 index 00000000..f7a71c94 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E1/abort-states.json @@ -0,0 +1,8 @@ +{ + "privatebin": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + } +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E1/compose-full.log b/documentation/audits/upgrade-spike-2026-09-06/evidence/E1/compose-full.log new file mode 100644 index 00000000..0577b5d0 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E1/compose-full.log @@ -0,0 +1,11 @@ +privatebin | execlineb: fatal: unable to exec /usr/sbin/php-fpm85: No such file or directory +privatebin | [06-Sep-2026 11:25:42] NOTICE: fpm is running, pid 12 +privatebin | [06-Sep-2026 11:25:42] NOTICE: ready to handle connections +privatebin | execlineb: fatal: unable to exec /usr/sbin/php-fpm85: No such file or directory +privatebin | execlineb: fatal: unable to exec /usr/sbin/php-fpm85: No such file or directory +privatebin | execlineb: fatal: unable to exec /usr/sbin/php-fpm85: No such file or directory +privatebin | execlineb: fatal: unable to exec /usr/sbin/php-fpm85: No such file or directory +privatebin | execlineb: fatal: unable to exec /usr/sbin/php-fpm85: No such file or directory +privatebin | 127.0.0.1 - - [06/Sep/2026:11:25:47 +0200] "GET / HTTP/1.1" 200 15538 "-" "Wget" "-" +privatebin | 172.18.0.1 - - [06/Sep/2026:11:25:47 +0200] "GET / HTTP/1.1" 200 19527 "-" "curl/8.14.1" "-" +privatebin | 172.18.0.1 - - [06/Sep/2026:11:25:47 +0200] "GET /?pasteid=b2bbc62b2d8ee417 HTTP/1.1" 200 336 "-" "curl/8.14.1" "-" diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E1/migration-lines.txt b/documentation/audits/upgrade-spike-2026-09-06/evidence/E1/migration-lines.txt new file mode 100644 index 00000000..e69de29b diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E1/run.log b/documentation/audits/upgrade-spike-2026-09-06/evidence/E1/run.log new file mode 100644 index 00000000..fb0585cd --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E1/run.log @@ -0,0 +1,14 @@ +[09:25:26] E1: deploying privatebin at FROM {'privatebin': 'privatebin/pdo:1.7.5'} +[09:25:35] FROM settled=True in 5.2s :: {"privatebin": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}} +[09:25:35] privatebin: seeded paste id=b2bbc62b2d8ee417 +[09:25:35] privatebin: readback http=200 marker_present=True +[09:25:35] C1 (seed reads back BEFORE): True +[09:25:35] E1: swapping to TO {'privatebin': 'privatebin/pdo:2.0.5'} +[09:25:36] TO up -d rc=0 +[09:25:41] TO settled=True in 5.2s :: {"privatebin": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}} +[09:25:41] migration lines observed: 0 +[09:25:41] privatebin: readback http=200 marker_present=True +[09:25:41] RESULT (seed reads back AFTER): True +[09:25:41] E1: ABORT — putting the FROM images back +[09:25:47] privatebin: readback http=200 marker_present=True +[09:25:47] ABORT: app came back in 5.2s; data present=True \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E1/to-states.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/E1/to-states.json new file mode 100644 index 00000000..f7a71c94 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E1/to-states.json @@ -0,0 +1,8 @@ +{ + "privatebin": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + } +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E1/verdict.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/E1/verdict.json new file mode 100644 index 00000000..79e62568 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E1/verdict.json @@ -0,0 +1,23 @@ +{ + "harness_version": 1, + "edge": "E1", + "app": "privatebin", + "note": "catalog transition cf8b645, a major", + "from": { + "privatebin": "privatebin/pdo:1.7.5" + }, + "to": { + "privatebin": "privatebin/pdo:2.0.5" + }, + "verdict": "proven", + "seed_read_before": true, + "seed_read_after": true, + "healthy_after": true, + "migration_observed": null, + "abort": "starts-and-serves", + "abort_detail": null, + "duration_s": 5.2, + "measured_at": "2026-09-06T09:25:47Z", + "evidence": "evidence/E1", + "total_s": 21.5 +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/abort-refusal.txt b/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/abort-refusal.txt new file mode 100644 index 00000000..931ba7ed --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/abort-refusal.txt @@ -0,0 +1,91 @@ +docmost-redis | 1:C 06 Sep 2026 11:26:52.375 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. +docmost | {"level":"error","time":"2026-09-06T09:29:34.884Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","msg":"Failed to run database migration. Exiting program."} +docmost | {"level":"error","time":"2026-09-06T09:29:34.884Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","err":{"type":"Error","message":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing","stack":"Error: corrupted migrations: previously executed migration 20260213T085259-notifications is missing\n at #ensureNoMissingMigrations (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:487:23)\n at #getState (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:439:40)\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n at async run (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:409:31)\n at async /app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/kysely.js:569:32\n at async DefaultConnectionProvider.provideConnection (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/driver/default-connection-provider.js:12:20)\n at async #migrate (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:268:20)\n at async MigrationService.migrateToLatest (/app/apps/server/dist/database/services/migration.service.js:36:36)\n at async DatabaseModule.onApplicationBootstrap (/app/apps/server/dist/database/database.module.js:50:13)\n at async callModuleBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/hooks/on-app-bootstrap.hook.js:51:9)\n at async NestApplication.callBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application-context.js:274:13)\n at async NestApplication.init (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:105:9)\n at async NestApplication.listen (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:175:13)\n at async bootstrap (/app/apps/server/dist/main.js:82:5)"},"msg":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing"} +docmost | /app/apps/server: +docmost |  ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL  server@0.25.3 start:prod: `cross-env NODE_ENV=production node dist/main` +docmost | Exit status 1 +docmost |  ELIFECYCLE  Command failed with exit code 1. +docmost | +docmost | > docmost@0.25.3 start /app +docmost | > pnpm --filter ./apps/server run start:prod +docmost | +docmost | +docmost | > server@0.25.3 start:prod /app/apps/server +docmost | > cross-env NODE_ENV=production node dist/main +docmost | +docmost | {"level":"info","time":"2026-09-06T09:30:06.457Z","pid":43,"hostname":"136cff373e66","context":"RedisModule","msg":"default: the connection was successfully established"} +docmost | {"level":"info","time":"2026-09-06T09:30:06.689Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Establishing database connection"} +docmost | {"level":"info","time":"2026-09-06T09:30:06.711Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Database connection successful"} +docmost | {"level":"error","time":"2026-09-06T09:30:06.830Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","msg":"Failed to run database migration. Exiting program."} +docmost | {"level":"error","time":"2026-09-06T09:30:06.830Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","err":{"type":"Error","message":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing","stack":"Error: corrupted migrations: previously executed migration 20260213T085259-notifications is missing\n at #ensureNoMissingMigrations (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:487:23)\n at #getState (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:439:40)\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n at async run (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:409:31)\n at async /app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/kysely.js:569:32\n at async DefaultConnectionProvider.provideConnection (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/driver/default-connection-provider.js:12:20)\n at async #migrate (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:268:20)\n at async MigrationService.migrateToLatest (/app/apps/server/dist/database/services/migration.service.js:36:36)\n at async DatabaseModule.onApplicationBootstrap (/app/apps/server/dist/database/database.module.js:50:13)\n at async callModuleBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/hooks/on-app-bootstrap.hook.js:51:9)\n at async NestApplication.callBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application-context.js:274:13)\n at async NestApplication.init (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:105:9)\n at async NestApplication.listen (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:175:13)\n at async bootstrap (/app/apps/server/dist/main.js:82:5)"},"msg":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing"} +docmost | /app/apps/server: +docmost |  ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL  server@0.25.3 start:prod: `cross-env NODE_ENV=production node dist/main` +docmost | Exit status 1 +docmost |  ELIFECYCLE  Command failed with exit code 1. +docmost | +docmost | > docmost@0.25.3 start /app +docmost | > pnpm --filter ./apps/server run start:prod +docmost | +docmost | +docmost | > server@0.25.3 start:prod /app/apps/server +docmost | > cross-env NODE_ENV=production node dist/main +docmost | +docmost | {"level":"info","time":"2026-09-06T09:31:03.944Z","pid":43,"hostname":"136cff373e66","context":"RedisModule","msg":"default: the connection was successfully established"} +docmost | {"level":"info","time":"2026-09-06T09:31:04.196Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Establishing database connection"} +docmost | {"level":"info","time":"2026-09-06T09:31:04.219Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Database connection successful"} +docmost | {"level":"error","time":"2026-09-06T09:31:04.343Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","msg":"Failed to run database migration. Exiting program."} +docmost | {"level":"error","time":"2026-09-06T09:31:04.343Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","err":{"type":"Error","message":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing","stack":"Error: corrupted migrations: previously executed migration 20260213T085259-notifications is missing\n at #ensureNoMissingMigrations (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:487:23)\n at #getState (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:439:40)\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n at async run (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:409:31)\n at async /app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/kysely.js:569:32\n at async DefaultConnectionProvider.provideConnection (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/driver/default-connection-provider.js:12:20)\n at async #migrate (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:268:20)\n at async MigrationService.migrateToLatest (/app/apps/server/dist/database/services/migration.service.js:36:36)\n at async DatabaseModule.onApplicationBootstrap (/app/apps/server/dist/database/database.module.js:50:13)\n at async callModuleBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/hooks/on-app-bootstrap.hook.js:51:9)\n at async NestApplication.callBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application-context.js:274:13)\n at async NestApplication.init (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:105:9)\n at async NestApplication.listen (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:175:13)\n at async bootstrap (/app/apps/server/dist/main.js:82:5)"},"msg":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing"} +docmost | /app/apps/server: +docmost |  ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL  server@0.25.3 start:prod: `cross-env NODE_ENV=production node dist/main` +docmost | Exit status 1 +docmost |  ELIFECYCLE  Command failed with exit code 1. +docmost-postgres | 2026-09-06 11:26:53.029 CEST [35] WARNING: no usable system locales were found +docmost-postgres | performing post-bootstrap initialization ... ok +docmost-postgres | initdb: warning: enabling "trust" authentication for local connections +docmost-postgres | initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. +docmost-postgres | syncing data to disk ... ok +docmost-postgres | +docmost-postgres | +docmost-postgres | Success. You can now start the database server using: +docmost-postgres | +docmost-postgres | pg_ctl -D /var/lib/postgresql/data -l logfile start +docmost-postgres | +docmost-postgres | waiting for server to start....2026-09-06 11:26:53.769 CEST [41] LOG: starting PostgreSQL 16.15 on x86_64-pc-linux-musl, compiled by gcc (Alpine 15.2.0) 15.2.0, 64-bit +docmost-postgres | 2026-09-06 11:26:53.772 CEST [41] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" +docmost-postgres | 2026-09-06 11:26:53.785 CEST [44] LOG: database system was shut down at 2026-09-06 11:26:53 CEST +docmost-postgres | 2026-09-06 11:26:53.794 CEST [41] LOG: database system is ready to accept connections +docmost-postgres | done +docmost-postgres | server started +docmost-postgres | CREATE DATABASE +docmost-postgres | +docmost-postgres | +docmost-postgres | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/* +docmost-postgres | +docmost-postgres | waiting for server to shut down....2026-09-06 11:26:53.946 CEST [41] LOG: received fast shutdown request +docmost-postgres | 2026-09-06 11:26:53.950 CEST [41] LOG: aborting any active transactions +docmost-postgres | 2026-09-06 11:26:53.953 CEST [41] LOG: background worker "logical replication launcher" (PID 47) exited with exit code 1 +docmost-postgres | 2026-09-06 11:26:53.955 CEST [42] LOG: shutting down +docmost-postgres | 2026-09-06 11:26:53.958 CEST [42] LOG: checkpoint starting: shutdown immediate +docmost-postgres | 2026-09-06 11:26:54.083 CEST [42] LOG: checkpoint complete: wrote 926 buffers (5.7%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.023 s, sync=0.089 s, total=0.128 s; sync files=301, longest=0.006 s, average=0.001 s; distance=4283 kB, estimate=4283 kB; lsn=0/1925D58, redo lsn=0/1925D58 +docmost-postgres | 2026-09-06 11:26:54.097 CEST [41] LOG: database system is shut down +docmost-postgres | done +docmost-postgres | server stopped +docmost-postgres | +docmost-postgres | PostgreSQL init process complete; ready for start up. +docmost-postgres | +docmost-postgres | 2026-09-06 11:26:54.181 CEST [1] LOG: starting PostgreSQL 16.15 on x86_64-pc-linux-musl, compiled by gcc (Alpine 15.2.0) 15.2.0, 64-bit +docmost-postgres | 2026-09-06 11:26:54.181 CEST [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 +docmost-postgres | 2026-09-06 11:26:54.181 CEST [1] LOG: listening on IPv6 address "::", port 5432 +docmost-postgres | 2026-09-06 11:26:54.188 CEST [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" +docmost-postgres | 2026-09-06 11:26:54.197 CEST [57] LOG: database system was shut down at 2026-09-06 11:26:54 CEST +docmost-postgres | 2026-09-06 11:26:54.207 CEST [1] LOG: database system is ready to accept connections +docmost-redis | 1:C 06 Sep 2026 11:26:52.375 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo +docmost-redis | 1:C 06 Sep 2026 11:26:52.375 * Redis version=7.4.11, bits=64, commit=00000000, modified=0, pid=1, just started +docmost-redis | 1:C 06 Sep 2026 11:26:52.375 * Configuration loaded +docmost-redis | 1:M 06 Sep 2026 11:26:52.376 * Increased maximum number of open files to 10032 (it was originally set to 1024). +docmost-redis | 1:M 06 Sep 2026 11:26:52.376 * monotonic clock: POSIX clock_gettime +docmost-redis | 1:M 06 Sep 2026 11:26:52.377 * Running mode=standalone, port=6379. +docmost-redis | 1:M 06 Sep 2026 11:26:52.377 * Server initialized +docmost-redis | 1:M 06 Sep 2026 11:26:52.381 * Creating AOF base file appendonly.aof.1.base.rdb on server start +docmost-redis | 1:M 06 Sep 2026 11:26:52.389 * Creating AOF incr file appendonly.aof.1.incr.aof on server start +docmost-redis | 1:M 06 Sep 2026 11:26:52.389 * Ready to accept connections tcp \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/abort-states.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/abort-states.json new file mode 100644 index 00000000..45ca6a7f --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/abort-states.json @@ -0,0 +1,20 @@ +{ + "docmost": { + "status": "restarting", + "health": "unhealthy", + "restarts": 0, + "exit": 1 + }, + "docmost-postgres": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + }, + "docmost-redis": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + } +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/compose-full.log b/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/compose-full.log new file mode 100644 index 00000000..0a959678 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/compose-full.log @@ -0,0 +1,256 @@ +docmost-redis | 1:C 06 Sep 2026 11:26:52.375 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. +docmost-redis | 1:C 06 Sep 2026 11:26:52.375 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo +docmost-redis | 1:C 06 Sep 2026 11:26:52.375 * Redis version=7.4.11, bits=64, commit=00000000, modified=0, pid=1, just started +docmost-redis | 1:C 06 Sep 2026 11:26:52.375 * Configuration loaded +docmost-redis | 1:M 06 Sep 2026 11:26:52.376 * Increased maximum number of open files to 10032 (it was originally set to 1024). +docmost-redis | 1:M 06 Sep 2026 11:26:52.376 * monotonic clock: POSIX clock_gettime +docmost-redis | 1:M 06 Sep 2026 11:26:52.377 * Running mode=standalone, port=6379. +docmost-redis | 1:M 06 Sep 2026 11:26:52.377 * Server initialized +docmost-redis | 1:M 06 Sep 2026 11:26:52.381 * Creating AOF base file appendonly.aof.1.base.rdb on server start +docmost-redis | 1:M 06 Sep 2026 11:26:52.389 * Creating AOF incr file appendonly.aof.1.incr.aof on server start +docmost-redis | 1:M 06 Sep 2026 11:26:52.389 * Ready to accept connections tcp +docmost-postgres | The files belonging to this database system will be owned by user "postgres". +docmost-postgres | This user must also own the server process. +docmost-postgres | +docmost-postgres | The database cluster will be initialized with locale "en_US.utf8". +docmost-postgres | The default database encoding has accordingly been set to "UTF8". +docmost-postgres | The default text search configuration will be set to "english". +docmost-postgres | +docmost-postgres | Data page checksums are disabled. +docmost-postgres | +docmost-postgres | fixing permissions on existing directory /var/lib/postgresql/data ... ok +docmost-postgres | creating subdirectories ... ok +docmost-postgres | selecting dynamic shared memory implementation ... posix +docmost-postgres | selecting default max_connections ... 100 +docmost-postgres | selecting default shared_buffers ... 128MB +docmost-postgres | selecting default time zone ... Europe/Budapest +docmost-postgres | creating configuration files ... ok +docmost-postgres | running bootstrap script ... ok +docmost-postgres | sh: locale: not found +docmost-postgres | 2026-09-06 11:26:53.029 CEST [35] WARNING: no usable system locales were found +docmost-postgres | performing post-bootstrap initialization ... ok +docmost-postgres | initdb: warning: enabling "trust" authentication for local connections +docmost-postgres | initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. +docmost-postgres | syncing data to disk ... ok +docmost-postgres | +docmost-postgres | +docmost-postgres | Success. You can now start the database server using: +docmost-postgres | +docmost-postgres | pg_ctl -D /var/lib/postgresql/data -l logfile start +docmost-postgres | +docmost-postgres | waiting for server to start....2026-09-06 11:26:53.769 CEST [41] LOG: starting PostgreSQL 16.15 on x86_64-pc-linux-musl, compiled by gcc (Alpine 15.2.0) 15.2.0, 64-bit +docmost-postgres | 2026-09-06 11:26:53.772 CEST [41] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" +docmost-postgres | 2026-09-06 11:26:53.785 CEST [44] LOG: database system was shut down at 2026-09-06 11:26:53 CEST +docmost-postgres | 2026-09-06 11:26:53.794 CEST [41] LOG: database system is ready to accept connections +docmost-postgres | done +docmost-postgres | server started +docmost-postgres | CREATE DATABASE +docmost-postgres | +docmost-postgres | +docmost-postgres | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/* +docmost-postgres | +docmost-postgres | waiting for server to shut down....2026-09-06 11:26:53.946 CEST [41] LOG: received fast shutdown request +docmost-postgres | 2026-09-06 11:26:53.950 CEST [41] LOG: aborting any active transactions +docmost-postgres | 2026-09-06 11:26:53.953 CEST [41] LOG: background worker "logical replication launcher" (PID 47) exited with exit code 1 +docmost-postgres | 2026-09-06 11:26:53.955 CEST [42] LOG: shutting down +docmost-postgres | 2026-09-06 11:26:53.958 CEST [42] LOG: checkpoint starting: shutdown immediate +docmost-postgres | 2026-09-06 11:26:54.083 CEST [42] LOG: checkpoint complete: wrote 926 buffers (5.7%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.023 s, sync=0.089 s, total=0.128 s; sync files=301, longest=0.006 s, average=0.001 s; distance=4283 kB, estimate=4283 kB; lsn=0/1925D58, redo lsn=0/1925D58 +docmost-postgres | 2026-09-06 11:26:54.097 CEST [41] LOG: database system is shut down +docmost-postgres | done +docmost-postgres | server stopped +docmost-postgres | +docmost-postgres | PostgreSQL init process complete; ready for start up. +docmost-postgres | +docmost-postgres | 2026-09-06 11:26:54.181 CEST [1] LOG: starting PostgreSQL 16.15 on x86_64-pc-linux-musl, compiled by gcc (Alpine 15.2.0) 15.2.0, 64-bit +docmost-postgres | 2026-09-06 11:26:54.181 CEST [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 +docmost-postgres | 2026-09-06 11:26:54.181 CEST [1] LOG: listening on IPv6 address "::", port 5432 +docmost-postgres | 2026-09-06 11:26:54.188 CEST [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" +docmost-postgres | 2026-09-06 11:26:54.197 CEST [57] LOG: database system was shut down at 2026-09-06 11:26:54 CEST +docmost-postgres | 2026-09-06 11:26:54.207 CEST [1] LOG: database system is ready to accept connections +docmost | +docmost | > docmost@0.25.3 start /app +docmost | > pnpm --filter ./apps/server run start:prod +docmost | +docmost | +docmost | > server@0.25.3 start:prod /app/apps/server +docmost | > cross-env NODE_ENV=production node dist/main +docmost | +docmost | {"level":"info","time":"2026-09-06T09:28:17.842Z","pid":43,"hostname":"136cff373e66","context":"RedisModule","msg":"default: the connection was successfully established"} +docmost | {"level":"info","time":"2026-09-06T09:28:18.127Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Establishing database connection"} +docmost | {"level":"info","time":"2026-09-06T09:28:18.149Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Database connection successful"} +docmost | {"level":"error","time":"2026-09-06T09:28:18.277Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","msg":"Failed to run database migration. Exiting program."} +docmost | {"level":"error","time":"2026-09-06T09:28:18.277Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","err":{"type":"Error","message":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing","stack":"Error: corrupted migrations: previously executed migration 20260213T085259-notifications is missing\n at #ensureNoMissingMigrations (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:487:23)\n at #getState (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:439:40)\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n at async run (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:409:31)\n at async /app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/kysely.js:569:32\n at async DefaultConnectionProvider.provideConnection (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/driver/default-connection-provider.js:12:20)\n at async #migrate (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:268:20)\n at async MigrationService.migrateToLatest (/app/apps/server/dist/database/services/migration.service.js:36:36)\n at async DatabaseModule.onApplicationBootstrap (/app/apps/server/dist/database/database.module.js:50:13)\n at async callModuleBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/hooks/on-app-bootstrap.hook.js:51:9)\n at async NestApplication.callBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application-context.js:274:13)\n at async NestApplication.init (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:105:9)\n at async NestApplication.listen (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:175:13)\n at async bootstrap (/app/apps/server/dist/main.js:82:5)"},"msg":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing"} +docmost | /app/apps/server: +docmost |  ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL  server@0.25.3 start:prod: `cross-env NODE_ENV=production node dist/main` +docmost | Exit status 1 +docmost |  ELIFECYCLE  Command failed with exit code 1. +docmost | +docmost | > docmost@0.25.3 start /app +docmost | > pnpm --filter ./apps/server run start:prod +docmost | +docmost | +docmost | > server@0.25.3 start:prod /app/apps/server +docmost | > cross-env NODE_ENV=production node dist/main +docmost | +docmost | {"level":"info","time":"2026-09-06T09:28:24.334Z","pid":43,"hostname":"136cff373e66","context":"RedisModule","msg":"default: the connection was successfully established"} +docmost | {"level":"info","time":"2026-09-06T09:28:24.581Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Establishing database connection"} +docmost | {"level":"info","time":"2026-09-06T09:28:24.613Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Database connection successful"} +docmost | {"level":"error","time":"2026-09-06T09:28:24.773Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","msg":"Failed to run database migration. Exiting program."} +docmost | {"level":"error","time":"2026-09-06T09:28:24.773Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","err":{"type":"Error","message":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing","stack":"Error: corrupted migrations: previously executed migration 20260213T085259-notifications is missing\n at #ensureNoMissingMigrations (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:487:23)\n at #getState (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:439:40)\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n at async run (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:409:31)\n at async /app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/kysely.js:569:32\n at async DefaultConnectionProvider.provideConnection (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/driver/default-connection-provider.js:12:20)\n at async #migrate (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:268:20)\n at async MigrationService.migrateToLatest (/app/apps/server/dist/database/services/migration.service.js:36:36)\n at async DatabaseModule.onApplicationBootstrap (/app/apps/server/dist/database/database.module.js:50:13)\n at async callModuleBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/hooks/on-app-bootstrap.hook.js:51:9)\n at async NestApplication.callBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application-context.js:274:13)\n at async NestApplication.init (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:105:9)\n at async NestApplication.listen (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:175:13)\n at async bootstrap (/app/apps/server/dist/main.js:82:5)"},"msg":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing"} +docmost | /app/apps/server: +docmost |  ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL  server@0.25.3 start:prod: `cross-env NODE_ENV=production node dist/main` +docmost | Exit status 1 +docmost |  ELIFECYCLE  Command failed with exit code 1. +docmost | +docmost | > docmost@0.25.3 start /app +docmost | > pnpm --filter ./apps/server run start:prod +docmost | +docmost | +docmost | > server@0.25.3 start:prod /app/apps/server +docmost | > cross-env NODE_ENV=production node dist/main +docmost | +docmost | {"level":"info","time":"2026-09-06T09:28:30.873Z","pid":43,"hostname":"136cff373e66","context":"RedisModule","msg":"default: the connection was successfully established"} +docmost | {"level":"info","time":"2026-09-06T09:28:31.148Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Establishing database connection"} +docmost | {"level":"info","time":"2026-09-06T09:28:31.169Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Database connection successful"} +docmost | {"level":"error","time":"2026-09-06T09:28:31.292Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","msg":"Failed to run database migration. Exiting program."} +docmost | {"level":"error","time":"2026-09-06T09:28:31.292Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","err":{"type":"Error","message":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing","stack":"Error: corrupted migrations: previously executed migration 20260213T085259-notifications is missing\n at #ensureNoMissingMigrations (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:487:23)\n at #getState (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:439:40)\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n at async run (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:409:31)\n at async /app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/kysely.js:569:32\n at async DefaultConnectionProvider.provideConnection (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/driver/default-connection-provider.js:12:20)\n at async #migrate (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:268:20)\n at async MigrationService.migrateToLatest (/app/apps/server/dist/database/services/migration.service.js:36:36)\n at async DatabaseModule.onApplicationBootstrap (/app/apps/server/dist/database/database.module.js:50:13)\n at async callModuleBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/hooks/on-app-bootstrap.hook.js:51:9)\n at async NestApplication.callBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application-context.js:274:13)\n at async NestApplication.init (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:105:9)\n at async NestApplication.listen (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:175:13)\n at async bootstrap (/app/apps/server/dist/main.js:82:5)"},"msg":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing"} +docmost | /app/apps/server: +docmost |  ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL  server@0.25.3 start:prod: `cross-env NODE_ENV=production node dist/main` +docmost | Exit status 1 +docmost |  ELIFECYCLE  Command failed with exit code 1. +docmost | +docmost | > docmost@0.25.3 start /app +docmost | > pnpm --filter ./apps/server run start:prod +docmost | +docmost | +docmost | > server@0.25.3 start:prod /app/apps/server +docmost | > cross-env NODE_ENV=production node dist/main +docmost | +docmost | {"level":"info","time":"2026-09-06T09:28:37.603Z","pid":43,"hostname":"136cff373e66","context":"RedisModule","msg":"default: the connection was successfully established"} +docmost | {"level":"info","time":"2026-09-06T09:28:37.835Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Establishing database connection"} +docmost | {"level":"info","time":"2026-09-06T09:28:37.860Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Database connection successful"} +docmost | {"level":"error","time":"2026-09-06T09:28:38.032Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","msg":"Failed to run database migration. Exiting program."} +docmost | {"level":"error","time":"2026-09-06T09:28:38.033Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","err":{"type":"Error","message":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing","stack":"Error: corrupted migrations: previously executed migration 20260213T085259-notifications is missing\n at #ensureNoMissingMigrations (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:487:23)\n at #getState (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:439:40)\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n at async run (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:409:31)\n at async /app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/kysely.js:569:32\n at async DefaultConnectionProvider.provideConnection (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/driver/default-connection-provider.js:12:20)\n at async #migrate (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:268:20)\n at async MigrationService.migrateToLatest (/app/apps/server/dist/database/services/migration.service.js:36:36)\n at async DatabaseModule.onApplicationBootstrap (/app/apps/server/dist/database/database.module.js:50:13)\n at async callModuleBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/hooks/on-app-bootstrap.hook.js:51:9)\n at async NestApplication.callBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application-context.js:274:13)\n at async NestApplication.init (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:105:9)\n at async NestApplication.listen (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:175:13)\n at async bootstrap (/app/apps/server/dist/main.js:82:5)"},"msg":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing"} +docmost | /app/apps/server: +docmost |  ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL  server@0.25.3 start:prod: `cross-env NODE_ENV=production node dist/main` +docmost | Exit status 1 +docmost |  ELIFECYCLE  Command failed with exit code 1. +docmost | +docmost | > docmost@0.25.3 start /app +docmost | > pnpm --filter ./apps/server run start:prod +docmost | +docmost | +docmost | > server@0.25.3 start:prod /app/apps/server +docmost | > cross-env NODE_ENV=production node dist/main +docmost | +docmost | {"level":"info","time":"2026-09-06T09:28:44.800Z","pid":43,"hostname":"136cff373e66","context":"RedisModule","msg":"default: the connection was successfully established"} +docmost | {"level":"info","time":"2026-09-06T09:28:45.047Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Establishing database connection"} +docmost | {"level":"info","time":"2026-09-06T09:28:45.073Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Database connection successful"} +docmost | {"level":"error","time":"2026-09-06T09:28:45.206Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","err":{"type":"Error","message":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing","stack":"Error: corrupted migrations: previously executed migration 20260213T085259-notifications is missing\n at #ensureNoMissingMigrations (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:487:23)\n at #getState (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:439:40)\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n at async run (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:409:31)\n at async /app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/kysely.js:569:32\n at async DefaultConnectionProvider.provideConnection (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/driver/default-connection-provider.js:12:20)\n at async #migrate (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:268:20)\n at async MigrationService.migrateToLatest (/app/apps/server/dist/database/services/migration.service.js:36:36)\n at async DatabaseModule.onApplicationBootstrap (/app/apps/server/dist/database/database.module.js:50:13)\n at async callModuleBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/hooks/on-app-bootstrap.hook.js:51:9)\n at async NestApplication.callBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application-context.js:274:13)\n at async NestApplication.init (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:105:9)\n at async NestApplication.listen (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:175:13)\n at async bootstrap (/app/apps/server/dist/main.js:82:5)"},"msg":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing"} +docmost | {"level":"error","time":"2026-09-06T09:28:45.206Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","msg":"Failed to run database migration. Exiting program."} +docmost | /app/apps/server: +docmost |  ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL  server@0.25.3 start:prod: `cross-env NODE_ENV=production node dist/main` +docmost | Exit status 1 +docmost |  ELIFECYCLE  Command failed with exit code 1. +docmost | +docmost | > docmost@0.25.3 start /app +docmost | > pnpm --filter ./apps/server run start:prod +docmost | +docmost | +docmost | > server@0.25.3 start:prod /app/apps/server +docmost | > cross-env NODE_ENV=production node dist/main +docmost | +docmost | {"level":"info","time":"2026-09-06T09:28:52.665Z","pid":43,"hostname":"136cff373e66","context":"RedisModule","msg":"default: the connection was successfully established"} +docmost | {"level":"info","time":"2026-09-06T09:28:52.895Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Establishing database connection"} +docmost | {"level":"info","time":"2026-09-06T09:28:52.922Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Database connection successful"} +docmost | {"level":"error","time":"2026-09-06T09:28:53.073Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","msg":"Failed to run database migration. Exiting program."} +docmost | {"level":"error","time":"2026-09-06T09:28:53.073Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","err":{"type":"Error","message":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing","stack":"Error: corrupted migrations: previously executed migration 20260213T085259-notifications is missing\n at #ensureNoMissingMigrations (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:487:23)\n at #getState (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:439:40)\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n at async run (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:409:31)\n at async /app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/kysely.js:569:32\n at async DefaultConnectionProvider.provideConnection (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/driver/default-connection-provider.js:12:20)\n at async #migrate (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:268:20)\n at async MigrationService.migrateToLatest (/app/apps/server/dist/database/services/migration.service.js:36:36)\n at async DatabaseModule.onApplicationBootstrap (/app/apps/server/dist/database/database.module.js:50:13)\n at async callModuleBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/hooks/on-app-bootstrap.hook.js:51:9)\n at async NestApplication.callBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application-context.js:274:13)\n at async NestApplication.init (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:105:9)\n at async NestApplication.listen (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:175:13)\n at async bootstrap (/app/apps/server/dist/main.js:82:5)"},"msg":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing"} +docmost | /app/apps/server: +docmost |  ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL  server@0.25.3 start:prod: `cross-env NODE_ENV=production node dist/main` +docmost | Exit status 1 +docmost |  ELIFECYCLE  Command failed with exit code 1. +docmost | +docmost | > docmost@0.25.3 start /app +docmost | > pnpm --filter ./apps/server run start:prod +docmost | +docmost | +docmost | > server@0.25.3 start:prod /app/apps/server +docmost | > cross-env NODE_ENV=production node dist/main +docmost | +docmost | {"level":"info","time":"2026-09-06T09:29:02.220Z","pid":43,"hostname":"136cff373e66","context":"RedisModule","msg":"default: the connection was successfully established"} +docmost | {"level":"info","time":"2026-09-06T09:29:02.465Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Establishing database connection"} +docmost | {"level":"info","time":"2026-09-06T09:29:02.487Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Database connection successful"} +docmost | {"level":"error","time":"2026-09-06T09:29:02.611Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","msg":"Failed to run database migration. Exiting program."} +docmost | {"level":"error","time":"2026-09-06T09:29:02.611Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","err":{"type":"Error","message":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing","stack":"Error: corrupted migrations: previously executed migration 20260213T085259-notifications is missing\n at #ensureNoMissingMigrations (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:487:23)\n at #getState (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:439:40)\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n at async run (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:409:31)\n at async /app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/kysely.js:569:32\n at async DefaultConnectionProvider.provideConnection (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/driver/default-connection-provider.js:12:20)\n at async #migrate (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:268:20)\n at async MigrationService.migrateToLatest (/app/apps/server/dist/database/services/migration.service.js:36:36)\n at async DatabaseModule.onApplicationBootstrap (/app/apps/server/dist/database/database.module.js:50:13)\n at async callModuleBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/hooks/on-app-bootstrap.hook.js:51:9)\n at async NestApplication.callBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application-context.js:274:13)\n at async NestApplication.init (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:105:9)\n at async NestApplication.listen (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:175:13)\n at async bootstrap (/app/apps/server/dist/main.js:82:5)"},"msg":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing"} +docmost | /app/apps/server: +docmost |  ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL  server@0.25.3 start:prod: `cross-env NODE_ENV=production node dist/main` +docmost | Exit status 1 +docmost |  ELIFECYCLE  Command failed with exit code 1. +docmost | +docmost | > docmost@0.25.3 start /app +docmost | > pnpm --filter ./apps/server run start:prod +docmost | +docmost | +docmost | > server@0.25.3 start:prod /app/apps/server +docmost | > cross-env NODE_ENV=production node dist/main +docmost | +docmost | {"level":"info","time":"2026-09-06T09:29:15.072Z","pid":43,"hostname":"136cff373e66","context":"RedisModule","msg":"default: the connection was successfully established"} +docmost | {"level":"info","time":"2026-09-06T09:29:15.375Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Establishing database connection"} +docmost | {"level":"info","time":"2026-09-06T09:29:15.399Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Database connection successful"} +docmost | {"level":"error","time":"2026-09-06T09:29:15.538Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","msg":"Failed to run database migration. Exiting program."} +docmost | {"level":"error","time":"2026-09-06T09:29:15.538Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","err":{"type":"Error","message":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing","stack":"Error: corrupted migrations: previously executed migration 20260213T085259-notifications is missing\n at #ensureNoMissingMigrations (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:487:23)\n at #getState (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:439:40)\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n at async run (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:409:31)\n at async /app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/kysely.js:569:32\n at async DefaultConnectionProvider.provideConnection (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/driver/default-connection-provider.js:12:20)\n at async #migrate (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:268:20)\n at async MigrationService.migrateToLatest (/app/apps/server/dist/database/services/migration.service.js:36:36)\n at async DatabaseModule.onApplicationBootstrap (/app/apps/server/dist/database/database.module.js:50:13)\n at async callModuleBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/hooks/on-app-bootstrap.hook.js:51:9)\n at async NestApplication.callBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application-context.js:274:13)\n at async NestApplication.init (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:105:9)\n at async NestApplication.listen (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:175:13)\n at async bootstrap (/app/apps/server/dist/main.js:82:5)"},"msg":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing"} +docmost | /app/apps/server: +docmost |  ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL  server@0.25.3 start:prod: `cross-env NODE_ENV=production node dist/main` +docmost | Exit status 1 +docmost |  ELIFECYCLE  Command failed with exit code 1. +docmost | +docmost | > docmost@0.25.3 start /app +docmost | > pnpm --filter ./apps/server run start:prod +docmost | +docmost | +docmost | > server@0.25.3 start:prod /app/apps/server +docmost | > cross-env NODE_ENV=production node dist/main +docmost | +docmost | {"level":"info","time":"2026-09-06T09:29:34.431Z","pid":43,"hostname":"136cff373e66","context":"RedisModule","msg":"default: the connection was successfully established"} +docmost | {"level":"info","time":"2026-09-06T09:29:34.695Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Establishing database connection"} +docmost | {"level":"info","time":"2026-09-06T09:29:34.716Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Database connection successful"} +docmost | {"level":"error","time":"2026-09-06T09:29:34.884Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","msg":"Failed to run database migration. Exiting program."} +docmost | {"level":"error","time":"2026-09-06T09:29:34.884Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","err":{"type":"Error","message":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing","stack":"Error: corrupted migrations: previously executed migration 20260213T085259-notifications is missing\n at #ensureNoMissingMigrations (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:487:23)\n at #getState (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:439:40)\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n at async run (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:409:31)\n at async /app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/kysely.js:569:32\n at async DefaultConnectionProvider.provideConnection (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/driver/default-connection-provider.js:12:20)\n at async #migrate (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:268:20)\n at async MigrationService.migrateToLatest (/app/apps/server/dist/database/services/migration.service.js:36:36)\n at async DatabaseModule.onApplicationBootstrap (/app/apps/server/dist/database/database.module.js:50:13)\n at async callModuleBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/hooks/on-app-bootstrap.hook.js:51:9)\n at async NestApplication.callBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application-context.js:274:13)\n at async NestApplication.init (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:105:9)\n at async NestApplication.listen (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:175:13)\n at async bootstrap (/app/apps/server/dist/main.js:82:5)"},"msg":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing"} +docmost | /app/apps/server: +docmost |  ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL  server@0.25.3 start:prod: `cross-env NODE_ENV=production node dist/main` +docmost | Exit status 1 +docmost |  ELIFECYCLE  Command failed with exit code 1. +docmost | +docmost | > docmost@0.25.3 start /app +docmost | > pnpm --filter ./apps/server run start:prod +docmost | +docmost | +docmost | > server@0.25.3 start:prod /app/apps/server +docmost | > cross-env NODE_ENV=production node dist/main +docmost | +docmost | {"level":"info","time":"2026-09-06T09:30:06.457Z","pid":43,"hostname":"136cff373e66","context":"RedisModule","msg":"default: the connection was successfully established"} +docmost | {"level":"info","time":"2026-09-06T09:30:06.689Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Establishing database connection"} +docmost | {"level":"info","time":"2026-09-06T09:30:06.711Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Database connection successful"} +docmost | {"level":"error","time":"2026-09-06T09:30:06.830Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","msg":"Failed to run database migration. Exiting program."} +docmost | {"level":"error","time":"2026-09-06T09:30:06.830Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","err":{"type":"Error","message":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing","stack":"Error: corrupted migrations: previously executed migration 20260213T085259-notifications is missing\n at #ensureNoMissingMigrations (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:487:23)\n at #getState (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:439:40)\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n at async run (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:409:31)\n at async /app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/kysely.js:569:32\n at async DefaultConnectionProvider.provideConnection (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/driver/default-connection-provider.js:12:20)\n at async #migrate (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:268:20)\n at async MigrationService.migrateToLatest (/app/apps/server/dist/database/services/migration.service.js:36:36)\n at async DatabaseModule.onApplicationBootstrap (/app/apps/server/dist/database/database.module.js:50:13)\n at async callModuleBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/hooks/on-app-bootstrap.hook.js:51:9)\n at async NestApplication.callBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application-context.js:274:13)\n at async NestApplication.init (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:105:9)\n at async NestApplication.listen (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:175:13)\n at async bootstrap (/app/apps/server/dist/main.js:82:5)"},"msg":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing"} +docmost | /app/apps/server: +docmost |  ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL  server@0.25.3 start:prod: `cross-env NODE_ENV=production node dist/main` +docmost | Exit status 1 +docmost |  ELIFECYCLE  Command failed with exit code 1. +docmost | +docmost | > docmost@0.25.3 start /app +docmost | > pnpm --filter ./apps/server run start:prod +docmost | +docmost | +docmost | > server@0.25.3 start:prod /app/apps/server +docmost | > cross-env NODE_ENV=production node dist/main +docmost | +docmost | {"level":"info","time":"2026-09-06T09:31:03.944Z","pid":43,"hostname":"136cff373e66","context":"RedisModule","msg":"default: the connection was successfully established"} +docmost | {"level":"info","time":"2026-09-06T09:31:04.196Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Establishing database connection"} +docmost | {"level":"info","time":"2026-09-06T09:31:04.219Z","pid":43,"hostname":"136cff373e66","context":"DatabaseModule","msg":"Database connection successful"} +docmost | {"level":"error","time":"2026-09-06T09:31:04.343Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","msg":"Failed to run database migration. Exiting program."} +docmost | {"level":"error","time":"2026-09-06T09:31:04.343Z","pid":43,"hostname":"136cff373e66","context":"DatabaseMigrationService","err":{"type":"Error","message":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing","stack":"Error: corrupted migrations: previously executed migration 20260213T085259-notifications is missing\n at #ensureNoMissingMigrations (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:487:23)\n at #getState (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:439:40)\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n at async run (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:409:31)\n at async /app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/kysely.js:569:32\n at async DefaultConnectionProvider.provideConnection (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/driver/default-connection-provider.js:12:20)\n at async #migrate (/app/node_modules/.pnpm/kysely@0.28.2/node_modules/kysely/dist/cjs/migration/migrator.js:268:20)\n at async MigrationService.migrateToLatest (/app/apps/server/dist/database/services/migration.service.js:36:36)\n at async DatabaseModule.onApplicationBootstrap (/app/apps/server/dist/database/database.module.js:50:13)\n at async callModuleBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/hooks/on-app-bootstrap.hook.js:51:9)\n at async NestApplication.callBootstrapHook (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application-context.js:274:13)\n at async NestApplication.init (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:105:9)\n at async NestApplication.listen (/app/node_modules/.pnpm/@nestjs+core@11.1.13_@nestjs+common@11.1.11_class-transformer@0.5.1_class-validator@0.1_12f3fd19d13f7701792c53839f3f8286/node_modules/@nestjs/core/nest-application.js:175:13)\n at async bootstrap (/app/apps/server/dist/main.js:82:5)"},"msg":"corrupted migrations: previously executed migration 20260213T085259-notifications is missing"} +docmost | /app/apps/server: +docmost |  ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL  server@0.25.3 start:prod: `cross-env NODE_ENV=production node dist/main` +docmost | Exit status 1 +docmost |  ELIFECYCLE  Command failed with exit code 1. diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/migration-lines.txt b/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/migration-lines.txt new file mode 100644 index 00000000..adb61f38 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/migration-lines.txt @@ -0,0 +1,6 @@ +docmost | {"level":"info","time":"2026-09-06T09:28:07.121Z","pid":45,"hostname":"a70ba98cb581","context":"DatabaseMigrationService","msg":"Migration \"20260213T085259-notifications\" executed successfully"} +docmost | {"level":"info","time":"2026-09-06T09:28:07.121Z","pid":45,"hostname":"a70ba98cb581","context":"DatabaseMigrationService","msg":"Migration \"20260213T085320-watchers\" executed successfully"} +docmost | {"level":"info","time":"2026-09-06T09:28:07.121Z","pid":45,"hostname":"a70ba98cb581","context":"DatabaseMigrationService","msg":"Migration \"20260213T085337-backfill-watchers\" executed successfully"} +docmost | {"level":"info","time":"2026-09-06T09:28:07.121Z","pid":45,"hostname":"a70ba98cb581","context":"DatabaseMigrationService","msg":"Migration \"20260224T233803-page-permissions\" executed successfully"} +docmost | {"level":"info","time":"2026-09-06T09:28:07.121Z","pid":45,"hostname":"a70ba98cb581","context":"DatabaseMigrationService","msg":"Migration \"20260228T223532-audit\" executed successfully"} +docmost | {"level":"info","time":"2026-09-06T09:28:07.121Z","pid":45,"hostname":"a70ba98cb581","context":"DatabaseMigrationService","msg":"Migration \"20260326T121350-user-sessions\" executed successfully"} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/run.log b/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/run.log new file mode 100644 index 00000000..6de952c5 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/run.log @@ -0,0 +1,13 @@ +[09:26:09] E2: deploying docmost at FROM {'docmost': 'docmost/docmost:0.25.3'} +[09:27:13] FROM settled=True in 10.7s :: {"docmost": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "docmost-postgres": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "docmost-redis": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}} +[09:27:13] docmost: /api/auth/setup http=200 rc=0 +[09:27:14] docmost: login as the seeded user http=200 ok=True +[09:27:14] C1 (seed reads back BEFORE): True +[09:27:14] E2: swapping to TO {'docmost': 'docmost/docmost:0.95.0'} +[09:27:59] TO up -d rc=0 +[09:28:09] TO settled=True in 10.7s :: {"docmost": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "docmost-postgres": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "docmost-redis": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}} +[09:28:10] migration lines observed: 6 +[09:28:10] docmost: login as the seeded user http=200 ok=True +[09:28:10] RESULT (seed reads back AFTER): True +[09:28:10] E2: ABORT — putting the FROM images back +[09:31:15] ABORT: the app did NOT come back (rc=0, 183.1s) \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/to-states.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/to-states.json new file mode 100644 index 00000000..c6bae4a5 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/to-states.json @@ -0,0 +1,20 @@ +{ + "docmost": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + }, + "docmost-postgres": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + }, + "docmost-redis": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + } +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/verdict.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/verdict.json new file mode 100644 index 00000000..9250373f --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E2/verdict.json @@ -0,0 +1,23 @@ +{ + "harness_version": 1, + "edge": "E2", + "app": "docmost", + "note": "catalog transition a2115b2; PostgreSQL constant across it", + "from": { + "docmost": "docmost/docmost:0.25.3" + }, + "to": { + "docmost": "docmost/docmost:0.95.0" + }, + "verdict": "proven", + "seed_read_before": true, + "seed_read_after": true, + "healthy_after": true, + "migration_observed": "docmost | {\"level\":\"info\",\"time\":\"2026-09-06T09:28:07.121Z\",\"pid\":45,\"hostname\":\"a70ba98cb581\",\"context\":\"DatabaseMigrationService\",\"msg\":\"Migration \\\"20260213T085259-notifications\\\" executed successfully\"}", + "abort": "refuses", + "abort_detail": "oO0Oo\ndocmost-redis | 1:C 06 Sep 2026 11:26:52.375 * Redis version=7.4.11, bits=64, commit=00000000, modified=0, pid=1, just started\ndocmost-redis | 1:C 06 Sep 2026 11:26:52.375 * Configuration loaded\ndocmost-redis | 1:M 06 Sep 2026 11:26:52.376 * Increased maximum number of open files to 10032 (it was originally set to 1024).\ndocmost-redis | 1:M 06 Sep 2026 11:26:52.376 * monotonic clock: POSIX clock_gettime\ndocmost-redis | 1:M 06 Sep 2026 11:26:52.377 * Running mode=standalone, port=6379.\ndocmost-redis | 1:M 06 Sep 2026 11:26:52.377 * Server initialized\ndocmost-redis | 1:M 06 Sep 2026 11:26:52.381 * Creating AOF base file appendonly.aof.1.base.rdb on server start\ndocmost-redis | 1:M 06 Sep 2026 11:26:52.389 * Creating AOF incr file appendonly.aof.1.incr.aof on server start\ndocmost-redis | 1:M 06 Sep 2026 11:26:52.389 * Ready to accept connections tcp", + "duration_s": 10.7, + "measured_at": "2026-09-06T09:31:15Z", + "evidence": "evidence/E2", + "total_s": 305.1 +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/abort-states.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/abort-states.json new file mode 100644 index 00000000..b910bf46 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/abort-states.json @@ -0,0 +1,14 @@ +{ + "bookstack": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + }, + "bookstack-db": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + } +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/compose-final.log b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/compose-final.log new file mode 100644 index 00000000..bd526560 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/compose-final.log @@ -0,0 +1,67 @@ +bookstack-db | 2026-09-06 11:39:24+02:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.6.2+maria~ubu2404 started. +bookstack-db | 2026-09-06 11:39:24+02:00 [Warn] [Entrypoint]: /sys/fs/cgroup///memory.pressure not writable, functionality unavailable to MariaDB +bookstack-db | 2026-09-06 11:39:24+02:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql' +bookstack-db | 2026-09-06 11:39:24+02:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.6.2+maria~ubu2404 started. +bookstack-db | 2026-09-06 11:39:25+02:00 [Note] [Entrypoint]: MariaDB upgrade not required +bookstack-db | 2026-09-06 11:39:25 0 [Note] Starting MariaDB 11.6.2-MariaDB-ubu2404 source revision d8dad8c3b54cd09fefce7bc3b9749f427eed9709 server_uid jIeFbcxMr8Qv1MJZcRmTuaJ7fpM= as process 1 +bookstack-db | 2026-09-06 11:39:25 0 [Note] InnoDB: Compressed tables use zlib 1.3 +bookstack-db | 2026-09-06 11:39:25 0 [Note] InnoDB: Number of transaction pools: 1 +bookstack-db | 2026-09-06 11:39:25 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions +bookstack-db | 2026-09-06 11:39:25 0 [Warning] mariadbd: io_uring_queue_init() failed with errno 0 +bookstack-db | 2026-09-06 11:39:25 0 [Warning] InnoDB: liburing disabled: falling back to innodb_use_native_aio=OFF +bookstack-db | 2026-09-06 11:39:25 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB +bookstack-db | 2026-09-06 11:39:25 0 [Note] InnoDB: Completed initialization of buffer pool +bookstack-db | 2026-09-06 11:39:25 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes) +bookstack-db | 2026-09-06 11:39:25 0 [Note] InnoDB: End of log at LSN=2242626 +bookstack-db | 2026-09-06 11:39:25 0 [Note] InnoDB: Opened 3 undo tablespaces +bookstack-db | 2026-09-06 11:39:25 0 [Note] InnoDB: 128 rollback segments in 3 undo tablespaces are active. +bookstack-db | 2026-09-06 11:39:25 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ... +bookstack-db | 2026-09-06 11:39:25 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB. +bookstack-db | 2026-09-06 11:39:25 0 [Note] InnoDB: log sequence number 2242626; transaction id 2917 +bookstack-db | 2026-09-06 11:39:25 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool +bookstack-db | 2026-09-06 11:39:25 0 [Note] Plugin 'FEEDBACK' is disabled. +bookstack-db | 2026-09-06 11:39:25 0 [Note] Plugin 'wsrep-provider' is disabled. +bookstack-db | 2026-09-06 11:39:25 0 [Note] InnoDB: Buffer pool(s) load completed at 260906 11:39:25 +bookstack-db | 2026-09-06 11:39:26 0 [Note] Server socket created on IP: '0.0.0.0'. +bookstack-db | 2026-09-06 11:39:26 0 [Note] Server socket created on IP: '::'. +bookstack-db | 2026-09-06 11:39:26 0 [Note] mariadbd: Event Scheduler: Loaded 0 events +bookstack-db | 2026-09-06 11:39:26 0 [Note] mariadbd: ready for connections. +bookstack | [migrations] started +bookstack | [migrations] 01-nginx-site-confs-default: skipped +bookstack | [migrations] 02-default-location: skipped +bookstack | [migrations] done +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | ██╗ ███████╗██╗ ██████╗ +bookstack | ██║ ██╔════╝██║██╔═══██╗ +bookstack | ██║ ███████╗██║██║ ██║ +bookstack | ██║ ╚════██║██║██║ ██║ +bookstack | ███████╗███████║██║╚██████╔╝ +bookstack | ╚══════╝╚══════╝╚═╝ ╚═════╝ +bookstack | +bookstack | Brought to you by linuxserver.io +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | To support LSIO projects visit: +bookstack | https://www.linuxserver.io/donate/ +bookstack | +bookstack | ─────────────────────────────────────── +bookstack | GID/UID +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | User UID: 1000 +bookstack | User GID: 1000 +bookstack | ─────────────────────────────────────── +bookstack | Linuxserver.io version: v25.02.2-ls202 +bookstack | Build-date: 2025-04-14T18:32:57+00:00 +bookstack | ─────────────────────────────────────── +bookstack | +bookstack-db | Version: '11.6.2-MariaDB-ubu2404' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution +bookstack-db | 2026-09-06 11:39:31 5 [Warning] Aborted connection 5 to db: 'unconnected' user: 'unauthenticated' host: '172.19.0.3' (This connection closed normally without authentication) +bookstack | using keys found in /config/keys +bookstack | Waiting for DB to be available +bookstack | +bookstack | INFO Nothing to migrate. +bookstack | +bookstack | [custom-init] No custom files found, skipping... +bookstack | [ls.io-init] done. diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/compose-full.log b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/compose-full.log new file mode 100644 index 00000000..56c9dbbe --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/compose-full.log @@ -0,0 +1,67 @@ +bookstack-db | 2026-09-06 11:36:57+02:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.6.2+maria~ubu2404 started. +bookstack-db | 2026-09-06 11:36:58+02:00 [Warn] [Entrypoint]: /sys/fs/cgroup///memory.pressure not writable, functionality unavailable to MariaDB +bookstack-db | 2026-09-06 11:36:58+02:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql' +bookstack-db | 2026-09-06 11:36:58+02:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.6.2+maria~ubu2404 started. +bookstack-db | 2026-09-06 11:36:58+02:00 [Note] [Entrypoint]: MariaDB upgrade not required +bookstack-db | 2026-09-06 11:36:58 0 [Note] Starting MariaDB 11.6.2-MariaDB-ubu2404 source revision d8dad8c3b54cd09fefce7bc3b9749f427eed9709 server_uid Bp04+pP2m99zSE//d9IaM/XEj0U= as process 1 +bookstack-db | 2026-09-06 11:36:58 0 [Note] InnoDB: Compressed tables use zlib 1.3 +bookstack-db | 2026-09-06 11:36:58 0 [Note] InnoDB: Number of transaction pools: 1 +bookstack-db | 2026-09-06 11:36:58 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions +bookstack-db | 2026-09-06 11:36:58 0 [Warning] mariadbd: io_uring_queue_init() failed with errno 0 +bookstack-db | 2026-09-06 11:36:58 0 [Warning] InnoDB: liburing disabled: falling back to innodb_use_native_aio=OFF +bookstack-db | 2026-09-06 11:36:58 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB +bookstack-db | 2026-09-06 11:36:58 0 [Note] InnoDB: Completed initialization of buffer pool +bookstack-db | 2026-09-06 11:36:58 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes) +bookstack-db | 2026-09-06 11:36:58 0 [Note] InnoDB: End of log at LSN=2309232 +bookstack-db | 2026-09-06 11:36:58 0 [Note] InnoDB: Opened 3 undo tablespaces +bookstack-db | 2026-09-06 11:36:58 0 [Note] InnoDB: 128 rollback segments in 3 undo tablespaces are active. +bookstack-db | 2026-09-06 11:36:58 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ... +bookstack-db | 2026-09-06 11:36:58 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB. +bookstack-db | 2026-09-06 11:36:58 0 [Note] InnoDB: log sequence number 2309232; transaction id 2925 +bookstack-db | 2026-09-06 11:36:58 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool +bookstack-db | 2026-09-06 11:36:58 0 [Note] Plugin 'FEEDBACK' is disabled. +bookstack-db | 2026-09-06 11:36:58 0 [Note] Plugin 'wsrep-provider' is disabled. +bookstack-db | 2026-09-06 11:36:58 0 [Note] InnoDB: Buffer pool(s) load completed at 260906 11:36:58 +bookstack-db | 2026-09-06 11:36:59 0 [Note] Server socket created on IP: '0.0.0.0'. +bookstack-db | 2026-09-06 11:36:59 0 [Note] Server socket created on IP: '::'. +bookstack-db | 2026-09-06 11:36:59 0 [Note] mariadbd: Event Scheduler: Loaded 0 events +bookstack-db | 2026-09-06 11:36:59 0 [Note] mariadbd: ready for connections. +bookstack-db | Version: '11.6.2-MariaDB-ubu2404' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution +bookstack-db | 2026-09-06 11:37:05 5 [Warning] Aborted connection 5 to db: 'unconnected' user: 'unauthenticated' host: '172.19.0.3' (This connection closed normally without authentication) +bookstack | [migrations] started +bookstack | [migrations] 01-nginx-site-confs-default: skipped +bookstack | [migrations] 02-default-location: skipped +bookstack | [migrations] done +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | ██╗ ███████╗██╗ ██████╗ +bookstack | ██║ ██╔════╝██║██╔═══██╗ +bookstack | ██║ ███████╗██║██║ ██║ +bookstack | ██║ ╚════██║██║██║ ██║ +bookstack | ███████╗███████║██║╚██████╔╝ +bookstack | ╚══════╝╚══════╝╚═╝ ╚═════╝ +bookstack | +bookstack | Brought to you by linuxserver.io +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | To support LSIO projects visit: +bookstack | https://www.linuxserver.io/donate/ +bookstack | +bookstack | ─────────────────────────────────────── +bookstack | GID/UID +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | User UID: 1000 +bookstack | User GID: 1000 +bookstack | ─────────────────────────────────────── +bookstack | Linuxserver.io version: v25.02.2-ls202 +bookstack | Build-date: 2025-04-14T18:32:57+00:00 +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | using keys found in /config/keys +bookstack | Waiting for DB to be available +bookstack | +bookstack | INFO Nothing to migrate. +bookstack | +bookstack | [custom-init] No custom files found, skipping... +bookstack | [ls.io-init] done. diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/migration-lines.txt b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/migration-lines.txt new file mode 100644 index 00000000..018c6aed --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/migration-lines.txt @@ -0,0 +1,6 @@ +bookstack-db | 2026-09-06 11:38:57+02:00 [Note] [Entrypoint]: MariaDB upgrade (mariadb-upgrade or creating healthcheck users) required, but skipped due to $MARIADB_AUTO_UPGRADE setting +bookstack | [migrations] started +bookstack | [migrations] 01-nginx-site-confs-default: skipped +bookstack | [migrations] 02-default-location: skipped +bookstack | [migrations] done +bookstack | INFO Running migrations. \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/run.log b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/run.log new file mode 100644 index 00000000..5e962ae3 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/run.log @@ -0,0 +1,14 @@ +[09:38:23] E3: deploying bookstack at FROM {'bookstack': 'lscr.io/linuxserver/bookstack:25.02.2', 'bookstack-db': 'mariadb:11.6'} +[09:38:50] FROM settled=True in 15.7s :: {"bookstack": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "bookstack-db": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}} +[09:38:51] bookstack: artisan create-admin rc=0 :: Admin account with email "spike-7cffb8f4@gate.invalid" successfully created! +[09:38:52] bookstack: readback of the seeded account found=True :: This will delete any configure multi-factor authentication methods for user: - ID: 3 - Name: spike-0505fd - Email: spike +[09:38:52] C1 (seed reads back BEFORE): True +[09:38:52] E3: swapping to TO {'bookstack': 'lscr.io/linuxserver/bookstack:26.05.2', 'bookstack-db': 'mariadb:12.3'} +[09:39:03] TO up -d rc=0 +[09:39:18] TO settled=True in 15.7s :: {"bookstack": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "bookstack-db": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}} +[09:39:18] migration lines observed: 6 +[09:39:19] bookstack: readback of the seeded account found=True :: This will delete any configure multi-factor authentication methods for user: - ID: 3 - Name: spike-0505fd - Email: spike +[09:39:19] RESULT (seed reads back AFTER): True +[09:39:19] E3: ABORT — putting the FROM images back +[09:39:41] bookstack: readback of the seeded account found=True :: This will delete any configure multi-factor authentication methods for user: - ID: 3 - Name: spike-0505fd - Email: spike +[09:39:41] ABORT: app came back in 10.5s; data present=True \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/to-full.log b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/to-full.log new file mode 100644 index 00000000..a896a0e7 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/to-full.log @@ -0,0 +1,95 @@ +bookstack | [migrations] started +bookstack-db | 2026-09-06 11:38:57+02:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:12.3.3+maria~ubu2404 started. +bookstack | [migrations] 01-nginx-site-confs-default: skipped +bookstack | [migrations] 02-default-location: skipped +bookstack | [migrations] done +bookstack | ─────────────────────────────────────── +bookstack-db | 2026-09-06 11:38:57+02:00 [Warn] [Entrypoint]: /sys/fs/cgroup///memory.pressure not writable, functionality unavailable to MariaDB +bookstack-db | 2026-09-06 11:38:57+02:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql' +bookstack-db | 2026-09-06 11:38:57+02:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:12.3.3+maria~ubu2404 started. +bookstack-db | 2026-09-06 11:38:57+02:00 [Note] [Entrypoint]: MariaDB upgrade (mariadb-upgrade or creating healthcheck users) required, but skipped due to $MARIADB_AUTO_UPGRADE setting +bookstack-db | 2026-09-06 11:38:57 0 [Note] Starting MariaDB 12.3.3-MariaDB-ubu2404 source revision 83e909fc2a0dbc394b4b683fb3fa2d7dcf26cc5e server_uid Q0SKRNuC+pTkInPJi+BZTvSvKBg= as process 1 +bookstack-db | 2026-09-06 11:38:57 0 [Note] InnoDB: Compressed tables use zlib 1.3 +bookstack-db | 2026-09-06 11:38:57 0 [Note] InnoDB: Number of transaction pools: 1 +bookstack-db | 2026-09-06 11:38:57 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions +bookstack-db | 2026-09-06 11:38:57 0 [Warning] mariadbd: io_uring_queue_init() failed with EPERM: sysctl kernel.io_uring_disabled has the value 2, or 1 and the user of the process is not a member of sysctl kernel.io_uring_group. (see man 2 io_uring_setup). +bookstack-db | create_uring failed: falling back to libaio +bookstack-db | 2026-09-06 11:38:57 0 [Note] InnoDB: Using Linux native AIO +bookstack-db | 2026-09-06 11:38:57 0 [Note] InnoDB: innodb_buffer_pool_size_max=8388608m, innodb_buffer_pool_size=128m +bookstack-db | 2026-09-06 11:38:57 0 [Note] InnoDB: Completed initialization of buffer pool +bookstack-db | 2026-09-06 11:38:57 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes) +bookstack-db | 2026-09-06 11:38:57 0 [Note] InnoDB: End of log at LSN=1142495 +bookstack-db | 2026-09-06 11:38:57 0 [Note] InnoDB: Opened 3 undo tablespaces +bookstack-db | 2026-09-06 11:38:57 0 [Note] InnoDB: 128 rollback segments in 3 undo tablespaces are active. +bookstack-db | 2026-09-06 11:38:57 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ... +bookstack-db | 2026-09-06 11:38:57 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB. +bookstack-db | 2026-09-06 11:38:57 0 [Note] InnoDB: log sequence number 1142495; transaction id 2268 +bookstack-db | 2026-09-06 11:38:57 0 [Note] Plugin 'FEEDBACK' is disabled. +bookstack-db | 2026-09-06 11:38:57 0 [Note] Plugin 'wsrep-provider' is disabled. +bookstack-db | 2026-09-06 11:38:57 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool +bookstack-db | 2026-09-06 11:38:57 0 [Note] InnoDB: Buffer pool(s) load completed at 260906 11:38:57 +bookstack-db | 2026-09-06 11:39:00 0 [Note] Server socket created on IP: '0.0.0.0', port: '3306'. +bookstack-db | 2026-09-06 11:39:00 0 [Note] Server socket created on IP: '::', port: '3306'. +bookstack-db | 2026-09-06 11:39:00 0 [Note] mariadbd: Event Scheduler: Loaded 0 events +bookstack | +bookstack | ██╗ ███████╗██╗ ██████╗ +bookstack | ██║ ██╔════╝██║██╔═══██╗ +bookstack | ██║ ███████╗██║██║ ██║ +bookstack | ██║ ╚════██║██║██║ ██║ +bookstack | ███████╗███████║██║╚██████╔╝ +bookstack | ╚══════╝╚══════╝╚═╝ ╚═════╝ +bookstack | +bookstack | Brought to you by linuxserver.io +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | To support the app dev(s) visit: +bookstack | Bookstack: https://www.bookstackapp.com/donate/ +bookstack | +bookstack | To support LSIO projects visit: +bookstack-db | 2026-09-06 11:39:00 0 [Note] mariadbd: ready for connections. +bookstack | https://www.linuxserver.io/donate/ +bookstack | +bookstack-db | Version: '12.3.3-MariaDB-ubu2404' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution +bookstack | ─────────────────────────────────────── +bookstack-db | 2026-09-06 11:39:04 5 [Warning] Aborted connection 5 to db: 'unconnected' user: 'unauthenticated' host: '172.19.0.3' (This connection closed normally without authentication) +bookstack | GID/UID +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | User UID: 1000 +bookstack | User GID: 1000 +bookstack | ─────────────────────────────────────── +bookstack | Linuxserver.io version: v26.05.2-ls276 +bookstack | Build-date: 2026-07-27T19:41:43+00:00 +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | using keys found in /config/keys +bookstack | **** The following active confs have different version dates than the samples that are shipped. **** +bookstack | **** This may be due to user customization or an update to the samples. **** +bookstack | **** You should compare the following files to the samples in the same folder and update them. **** +bookstack | **** Use the link at the top of the file to view the changelog. **** +bookstack | ┌────────────┬────────────┬────────────────────────────────────────────────────────────────────────┐ +bookstack | │ old date │ new date │ path │ +bookstack | ├────────────┼────────────┼────────────────────────────────────────────────────────────────────────┤ +bookstack | │ 2024-12-06 │ 2026-06-27 │ /config/nginx/ssl.conf │ +bookstack | │ 2024-12-17 │ 2025-12-26 │ /config/nginx/nginx.conf │ +bookstack | │ 2024-07-16 │ 2026-06-27 │ /config/nginx/site-confs/default.conf │ +bookstack | └────────────┴────────────┴────────────────────────────────────────────────────────────────────────┘ +bookstack | Waiting for DB to be available +bookstack | +bookstack | INFO Running migrations. +bookstack | +bookstack | 2025_04_18_215145_add_content_refs_and_archived_to_comments ... 84.79ms DONE +bookstack | 2025_09_02_111542_remove_unused_columns ....................... 77.77ms DONE +bookstack | 2025_09_15_132850_create_entities_table ...................... 296.01ms DONE +bookstack | 2025_09_15_134701_migrate_entity_data ......................... 13.66ms DONE +bookstack | 2025_09_15_134751_update_entity_relation_columns ............. 862.86ms DONE +bookstack | 2025_09_15_134813_drop_old_entity_tables ...................... 49.37ms DONE +bookstack | 2025_10_18_163331_clean_user_id_references ................... 450.74ms DONE +bookstack | 2025_10_22_134507_update_comments_relation_field_names ........ 46.69ms DONE +bookstack | 2025_11_23_161812_create_slug_history_table .................. 128.99ms DONE +bookstack | 2025_12_15_140219_create_mention_history_table ................ 71.34ms DONE +bookstack | 2025_12_19_103417_add_views_viewable_type_index ............... 27.87ms DONE +bookstack | 2026_04_19_141616_add_revision_view_all_permission ............. 6.76ms DONE +bookstack | +bookstack | [custom-init] No custom files found, skipping... +bookstack | [ls.io-init] done. diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/to-states.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/to-states.json new file mode 100644 index 00000000..b910bf46 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/to-states.json @@ -0,0 +1,14 @@ +{ + "bookstack": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + }, + "bookstack-db": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + } +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/verdict.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/verdict.json new file mode 100644 index 00000000..71347e7f --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3/verdict.json @@ -0,0 +1,25 @@ +{ + "harness_version": 1, + "edge": "E3", + "app": "bookstack", + "note": "catalog transition 0b73e5e: app AND engine together", + "from": { + "bookstack": "lscr.io/linuxserver/bookstack:25.02.2", + "bookstack-db": "mariadb:11.6" + }, + "to": { + "bookstack": "lscr.io/linuxserver/bookstack:26.05.2", + "bookstack-db": "mariadb:12.3" + }, + "verdict": "proven", + "seed_read_before": true, + "seed_read_after": true, + "healthy_after": true, + "migration_observed": "bookstack-db | 2026-09-06 11:38:57+02:00 [Note] [Entrypoint]: MariaDB upgrade (mariadb-upgrade or creating healthcheck users) required, but skipped due to $MARIADB_AUTO_UPGRADE setting", + "abort": "starts-and-serves", + "abort_detail": null, + "duration_s": 15.7, + "measured_at": "2026-09-06T09:39:41Z", + "evidence": "evidence/E3", + "total_s": 77.5 +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/abort-states.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/abort-states.json new file mode 100644 index 00000000..b910bf46 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/abort-states.json @@ -0,0 +1,14 @@ +{ + "bookstack": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + }, + "bookstack-db": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + } +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/compose-final.log b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/compose-final.log new file mode 100644 index 00000000..91166479 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/compose-final.log @@ -0,0 +1,114 @@ +bookstack | [migrations] started +bookstack | [migrations] 01-nginx-site-confs-default: skipped +bookstack | [migrations] 02-default-location: skipped +bookstack | [migrations] done +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | ██╗ ███████╗██╗ ██████╗ +bookstack | ██║ ██╔════╝██║██╔═══██╗ +bookstack | ██║ ███████╗██║██║ ██║ +bookstack | ██║ ╚════██║██║██║ ██║ +bookstack | ███████╗███████║██║╚██████╔╝ +bookstack | ╚══════╝╚══════╝╚═╝ ╚═════╝ +bookstack | +bookstack | Brought to you by linuxserver.io +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | To support LSIO projects visit: +bookstack | https://www.linuxserver.io/donate/ +bookstack | +bookstack | ─────────────────────────────────────── +bookstack-db | 2026-09-06 11:39:46+02:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.6.2+maria~ubu2404 started. +bookstack | GID/UID +bookstack-db | 2026-09-06 11:39:46+02:00 [Warn] [Entrypoint]: /sys/fs/cgroup///memory.pressure not writable, functionality unavailable to MariaDB +bookstack-db | 2026-09-06 11:39:46+02:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql' +bookstack-db | 2026-09-06 11:39:46+02:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.6.2+maria~ubu2404 started. +bookstack-db | 2026-09-06 11:39:46+02:00 [Note] [Entrypoint]: Initializing database files +bookstack-db | 2026-09-06 11:39:46 0 [Warning] mariadbd: io_uring_queue_init() failed with errno 2 +bookstack-db | 2026-09-06 11:39:46 0 [Warning] InnoDB: liburing disabled: falling back to innodb_use_native_aio=OFF +bookstack-db | 2026-09-06 11:39:49+02:00 [Note] [Entrypoint]: Database files initialized +bookstack-db | 2026-09-06 11:39:49+02:00 [Note] [Entrypoint]: Starting temporary server +bookstack-db | 2026-09-06 11:39:49+02:00 [Note] [Entrypoint]: Waiting for server startup +bookstack-db | 2026-09-06 11:39:49 0 [Note] Starting MariaDB 11.6.2-MariaDB-ubu2404 source revision d8dad8c3b54cd09fefce7bc3b9749f427eed9709 server_uid 5VZbo5kSbfrs3TIzpdyzN2xzF/E= as process 98 +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: Compressed tables use zlib 1.3 +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: Number of transaction pools: 1 +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions +bookstack-db | 2026-09-06 11:39:49 0 [Warning] mariadbd: io_uring_queue_init() failed with errno 0 +bookstack-db | 2026-09-06 11:39:49 0 [Warning] InnoDB: liburing disabled: falling back to innodb_use_native_aio=OFF +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: Completed initialization of buffer pool +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes) +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: End of log at LSN=47747 +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: Opened 3 undo tablespaces +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: 128 rollback segments in 3 undo tablespaces are active. +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ... +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB. +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: log sequence number 47747; transaction id 14 +bookstack-db | 2026-09-06 11:39:49 0 [Note] Plugin 'FEEDBACK' is disabled. +bookstack-db | 2026-09-06 11:39:49 0 [Note] Plugin 'wsrep-provider' is disabled. +bookstack-db | 2026-09-06 11:39:53 0 [Note] mariadbd: Event Scheduler: Loaded 0 events +bookstack-db | 2026-09-06 11:39:53 0 [Note] mariadbd: ready for connections. +bookstack-db | Version: '11.6.2-MariaDB-ubu2404' socket: '/run/mysqld/mysqld.sock' port: 0 mariadb.org binary distribution +bookstack-db | 2026-09-06 11:39:54+02:00 [Note] [Entrypoint]: Temporary server started. +bookstack-db | 2026-09-06 11:39:54+02:00 [Note] [Entrypoint]: Creating database bookstack +bookstack-db | 2026-09-06 11:39:54+02:00 [Note] [Entrypoint]: Creating user bookstack +bookstack-db | 2026-09-06 11:39:54+02:00 [Note] [Entrypoint]: Giving user bookstack access to schema bookstack +bookstack-db | 2026-09-06 11:39:54+02:00 [Note] [Entrypoint]: Securing system users (equivalent to running mysql_secure_installation) +bookstack-db | +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | User UID: 1000 +bookstack | User GID: 1000 +bookstack | ─────────────────────────────────────── +bookstack | Linuxserver.io version: v25.02.2-ls202 +bookstack | Build-date: 2025-04-14T18:32:57+00:00 +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | using keys found in /config/keys +bookstack | Waiting for DB to be available +bookstack | +bookstack | INFO Nothing to migrate. +bookstack | +bookstack | [custom-init] No custom files found, skipping... +bookstack | [ls.io-init] done. +bookstack-db | 2026-09-06 11:39:55+02:00 [Note] [Entrypoint]: Stopping temporary server +bookstack-db | 2026-09-06 11:39:55 0 [Note] mariadbd (initiated by: unknown): Normal shutdown +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: FTS optimize thread exiting. +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Starting shutdown... +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Buffer pool(s) dump completed at 260906 11:39:55 +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1" +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Shutdown completed; log sequence number 47747; transaction id 15 +bookstack-db | 2026-09-06 11:39:55 0 [Note] mariadbd: Shutdown complete +bookstack-db | +bookstack-db | 2026-09-06 11:39:55+02:00 [Note] [Entrypoint]: Temporary server stopped +bookstack-db | +bookstack-db | 2026-09-06 11:39:55+02:00 [Note] [Entrypoint]: MariaDB init process done. Ready for start up. +bookstack-db | +bookstack-db | 2026-09-06 11:39:55 0 [Note] Starting MariaDB 11.6.2-MariaDB-ubu2404 source revision d8dad8c3b54cd09fefce7bc3b9749f427eed9709 server_uid 5VZbo5kSbfrs3TIzpdyzN2xzF/E= as process 1 +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Compressed tables use zlib 1.3 +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Number of transaction pools: 1 +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions +bookstack-db | 2026-09-06 11:39:55 0 [Warning] mariadbd: io_uring_queue_init() failed with errno 0 +bookstack-db | 2026-09-06 11:39:55 0 [Warning] InnoDB: liburing disabled: falling back to innodb_use_native_aio=OFF +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Completed initialization of buffer pool +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes) +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: End of log at LSN=47747 +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Opened 3 undo tablespaces +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: 128 rollback segments in 3 undo tablespaces are active. +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ... +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB. +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: log sequence number 47747; transaction id 14 +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool +bookstack-db | 2026-09-06 11:39:55 0 [Note] Plugin 'FEEDBACK' is disabled. +bookstack-db | 2026-09-06 11:39:55 0 [Note] Plugin 'wsrep-provider' is disabled. +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Buffer pool(s) load completed at 260906 11:39:55 +bookstack-db | 2026-09-06 11:39:56 0 [Note] Server socket created on IP: '0.0.0.0'. +bookstack-db | 2026-09-06 11:39:56 0 [Note] Server socket created on IP: '::'. +bookstack-db | 2026-09-06 11:39:56 0 [Note] mariadbd: Event Scheduler: Loaded 0 events +bookstack-db | 2026-09-06 11:39:56 0 [Note] mariadbd: ready for connections. +bookstack-db | Version: '11.6.2-MariaDB-ubu2404' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution +bookstack-db | 2026-09-06 11:40:03 5 [Warning] Aborted connection 5 to db: 'unconnected' user: 'unauthenticated' host: '172.19.0.3' (This connection closed normally without authentication) +bookstack-db | 2026-09-06 11:40:25 17 [Warning] Aborted connection 17 to db: 'unconnected' user: 'unauthenticated' host: '172.19.0.3' (This connection closed normally without authentication) +bookstack-db | 2026-09-06 11:40:47 27 [Warning] Aborted connection 27 to db: 'unconnected' user: 'unauthenticated' host: '172.19.0.3' (This connection closed normally without authentication) diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/migration-lines.txt b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/migration-lines.txt new file mode 100644 index 00000000..cb5b6e16 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/migration-lines.txt @@ -0,0 +1,6 @@ +bookstack | [migrations] started +bookstack | [migrations] 01-nginx-site-confs-default: skipped +bookstack | [migrations] 02-default-location: skipped +bookstack | [migrations] done +bookstack | INFO Running migrations. +bookstack | 2025_09_15_134701_migrate_entity_data ......................... 12.89ms DONE \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/run.log b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/run.log new file mode 100644 index 00000000..1ef46797 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/run.log @@ -0,0 +1,14 @@ +[09:39:45] E3a: deploying bookstack at FROM {'bookstack': 'lscr.io/linuxserver/bookstack:25.02.2', 'bookstack-db': 'mariadb:11.6'} +[09:40:17] FROM settled=True in 15.7s :: {"bookstack": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "bookstack-db": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}} +[09:40:18] bookstack: artisan create-admin rc=0 :: Admin account with email "spike-f5408487@gate.invalid" successfully created! +[09:40:19] bookstack: readback of the seeded account found=True :: This will delete any configure multi-factor authentication methods for user: - ID: 3 - Name: spike-2e6215 - Email: spike +[09:40:19] C1 (seed reads back BEFORE): True +[09:40:19] E3a: swapping to TO {'bookstack': 'lscr.io/linuxserver/bookstack:26.05.2', 'bookstack-db': 'mariadb:11.6'} +[09:40:24] TO up -d rc=0 +[09:40:40] TO settled=True in 15.7s :: {"bookstack": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "bookstack-db": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}} +[09:40:40] migration lines observed: 6 +[09:40:41] bookstack: readback of the seeded account found=True :: This will delete any configure multi-factor authentication methods for user: - ID: 3 - Name: spike-2e6215 - Email: spike +[09:40:41] RESULT (seed reads back AFTER): True +[09:40:41] E3a: ABORT — putting the FROM images back +[09:40:57] bookstack: readback of the seeded account found=True :: This will delete any configure multi-factor authentication methods for user: - ID: 3 - Name: spike-2e6215 - Email: spike +[09:40:57] ABORT: app came back in 10.5s; data present=True \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/to-full.log b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/to-full.log new file mode 100644 index 00000000..d3a0b546 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/to-full.log @@ -0,0 +1,140 @@ +bookstack | [migrations] started +bookstack | [migrations] 01-nginx-site-confs-default: skipped +bookstack | [migrations] 02-default-location: skipped +bookstack-db | 2026-09-06 11:39:46+02:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.6.2+maria~ubu2404 started. +bookstack-db | 2026-09-06 11:39:46+02:00 [Warn] [Entrypoint]: /sys/fs/cgroup///memory.pressure not writable, functionality unavailable to MariaDB +bookstack-db | 2026-09-06 11:39:46+02:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql' +bookstack-db | 2026-09-06 11:39:46+02:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.6.2+maria~ubu2404 started. +bookstack-db | 2026-09-06 11:39:46+02:00 [Note] [Entrypoint]: Initializing database files +bookstack-db | 2026-09-06 11:39:46 0 [Warning] mariadbd: io_uring_queue_init() failed with errno 2 +bookstack-db | 2026-09-06 11:39:46 0 [Warning] InnoDB: liburing disabled: falling back to innodb_use_native_aio=OFF +bookstack-db | 2026-09-06 11:39:49+02:00 [Note] [Entrypoint]: Database files initialized +bookstack-db | 2026-09-06 11:39:49+02:00 [Note] [Entrypoint]: Starting temporary server +bookstack-db | 2026-09-06 11:39:49+02:00 [Note] [Entrypoint]: Waiting for server startup +bookstack-db | 2026-09-06 11:39:49 0 [Note] Starting MariaDB 11.6.2-MariaDB-ubu2404 source revision d8dad8c3b54cd09fefce7bc3b9749f427eed9709 server_uid 5VZbo5kSbfrs3TIzpdyzN2xzF/E= as process 98 +bookstack | [migrations] done +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | ██╗ ███████╗██╗ ██████╗ +bookstack | ██║ ██╔════╝██║██╔═══██╗ +bookstack | ██║ ███████╗██║██║ ██║ +bookstack | ██║ ╚════██║██║██║ ██║ +bookstack | ███████╗███████║██║╚██████╔╝ +bookstack | ╚══════╝╚══════╝╚═╝ ╚═════╝ +bookstack | +bookstack | Brought to you by linuxserver.io +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | To support the app dev(s) visit: +bookstack | Bookstack: https://www.bookstackapp.com/donate/ +bookstack | +bookstack | To support LSIO projects visit: +bookstack | https://www.linuxserver.io/donate/ +bookstack | +bookstack | ─────────────────────────────────────── +bookstack | GID/UID +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | User UID: 1000 +bookstack | User GID: 1000 +bookstack | ─────────────────────────────────────── +bookstack | Linuxserver.io version: v26.05.2-ls276 +bookstack | Build-date: 2026-07-27T19:41:43+00:00 +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | using keys found in /config/keys +bookstack | **** The following active confs have different version dates than the samples that are shipped. **** +bookstack | **** This may be due to user customization or an update to the samples. **** +bookstack | **** You should compare the following files to the samples in the same folder and update them. **** +bookstack | **** Use the link at the top of the file to view the changelog. **** +bookstack | ┌────────────┬────────────┬────────────────────────────────────────────────────────────────────────┐ +bookstack | │ old date │ new date │ path │ +bookstack | ├────────────┼────────────┼────────────────────────────────────────────────────────────────────────┤ +bookstack | │ 2024-12-06 │ 2026-06-27 │ /config/nginx/ssl.conf │ +bookstack | │ 2024-12-17 │ 2025-12-26 │ /config/nginx/nginx.conf │ +bookstack | │ 2024-07-16 │ 2026-06-27 │ /config/nginx/site-confs/default.conf │ +bookstack | └────────────┴────────────┴────────────────────────────────────────────────────────────────────────┘ +bookstack | Waiting for DB to be available +bookstack | +bookstack | INFO Running migrations. +bookstack | +bookstack | 2025_04_18_215145_add_content_refs_and_archived_to_comments ... 65.54ms DONE +bookstack | 2025_09_02_111542_remove_unused_columns ....................... 65.06ms DONE +bookstack | 2025_09_15_132850_create_entities_table ...................... 271.56ms DONE +bookstack | 2025_09_15_134701_migrate_entity_data ......................... 12.89ms DONE +bookstack | 2025_09_15_134751_update_entity_relation_columns ............. 797.09ms DONE +bookstack | 2025_09_15_134813_drop_old_entity_tables ...................... 47.54ms DONE +bookstack | 2025_10_18_163331_clean_user_id_references ................... 453.35ms DONE +bookstack | 2025_10_22_134507_update_comments_relation_field_names ........ 35.20ms DONE +bookstack | 2025_11_23_161812_create_slug_history_table .................. 114.64ms DONE +bookstack | 2025_12_15_140219_create_mention_history_table ................ 60.44ms DONE +bookstack | 2025_12_19_103417_add_views_viewable_type_index ............... 24.37ms DONE +bookstack | 2026_04_19_141616_add_revision_view_all_permission ............. 4.73ms DONE +bookstack | +bookstack | [custom-init] No custom files found, skipping... +bookstack | [ls.io-init] done. +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: Compressed tables use zlib 1.3 +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: Number of transaction pools: 1 +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions +bookstack-db | 2026-09-06 11:39:49 0 [Warning] mariadbd: io_uring_queue_init() failed with errno 0 +bookstack-db | 2026-09-06 11:39:49 0 [Warning] InnoDB: liburing disabled: falling back to innodb_use_native_aio=OFF +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: Completed initialization of buffer pool +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes) +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: End of log at LSN=47747 +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: Opened 3 undo tablespaces +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: 128 rollback segments in 3 undo tablespaces are active. +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ... +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB. +bookstack-db | 2026-09-06 11:39:49 0 [Note] InnoDB: log sequence number 47747; transaction id 14 +bookstack-db | 2026-09-06 11:39:49 0 [Note] Plugin 'FEEDBACK' is disabled. +bookstack-db | 2026-09-06 11:39:49 0 [Note] Plugin 'wsrep-provider' is disabled. +bookstack-db | 2026-09-06 11:39:53 0 [Note] mariadbd: Event Scheduler: Loaded 0 events +bookstack-db | 2026-09-06 11:39:53 0 [Note] mariadbd: ready for connections. +bookstack-db | Version: '11.6.2-MariaDB-ubu2404' socket: '/run/mysqld/mysqld.sock' port: 0 mariadb.org binary distribution +bookstack-db | 2026-09-06 11:39:54+02:00 [Note] [Entrypoint]: Temporary server started. +bookstack-db | 2026-09-06 11:39:54+02:00 [Note] [Entrypoint]: Creating database bookstack +bookstack-db | 2026-09-06 11:39:54+02:00 [Note] [Entrypoint]: Creating user bookstack +bookstack-db | 2026-09-06 11:39:54+02:00 [Note] [Entrypoint]: Giving user bookstack access to schema bookstack +bookstack-db | 2026-09-06 11:39:54+02:00 [Note] [Entrypoint]: Securing system users (equivalent to running mysql_secure_installation) +bookstack-db | +bookstack-db | 2026-09-06 11:39:55+02:00 [Note] [Entrypoint]: Stopping temporary server +bookstack-db | 2026-09-06 11:39:55 0 [Note] mariadbd (initiated by: unknown): Normal shutdown +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: FTS optimize thread exiting. +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Starting shutdown... +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Buffer pool(s) dump completed at 260906 11:39:55 +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1" +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Shutdown completed; log sequence number 47747; transaction id 15 +bookstack-db | 2026-09-06 11:39:55 0 [Note] mariadbd: Shutdown complete +bookstack-db | +bookstack-db | 2026-09-06 11:39:55+02:00 [Note] [Entrypoint]: Temporary server stopped +bookstack-db | +bookstack-db | 2026-09-06 11:39:55+02:00 [Note] [Entrypoint]: MariaDB init process done. Ready for start up. +bookstack-db | +bookstack-db | 2026-09-06 11:39:55 0 [Note] Starting MariaDB 11.6.2-MariaDB-ubu2404 source revision d8dad8c3b54cd09fefce7bc3b9749f427eed9709 server_uid 5VZbo5kSbfrs3TIzpdyzN2xzF/E= as process 1 +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Compressed tables use zlib 1.3 +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Number of transaction pools: 1 +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions +bookstack-db | 2026-09-06 11:39:55 0 [Warning] mariadbd: io_uring_queue_init() failed with errno 0 +bookstack-db | 2026-09-06 11:39:55 0 [Warning] InnoDB: liburing disabled: falling back to innodb_use_native_aio=OFF +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Completed initialization of buffer pool +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes) +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: End of log at LSN=47747 +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Opened 3 undo tablespaces +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: 128 rollback segments in 3 undo tablespaces are active. +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ... +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB. +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: log sequence number 47747; transaction id 14 +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool +bookstack-db | 2026-09-06 11:39:55 0 [Note] Plugin 'FEEDBACK' is disabled. +bookstack-db | 2026-09-06 11:39:55 0 [Note] Plugin 'wsrep-provider' is disabled. +bookstack-db | 2026-09-06 11:39:55 0 [Note] InnoDB: Buffer pool(s) load completed at 260906 11:39:55 +bookstack-db | 2026-09-06 11:39:56 0 [Note] Server socket created on IP: '0.0.0.0'. +bookstack-db | 2026-09-06 11:39:56 0 [Note] Server socket created on IP: '::'. +bookstack-db | 2026-09-06 11:39:56 0 [Note] mariadbd: Event Scheduler: Loaded 0 events +bookstack-db | 2026-09-06 11:39:56 0 [Note] mariadbd: ready for connections. +bookstack-db | Version: '11.6.2-MariaDB-ubu2404' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution +bookstack-db | 2026-09-06 11:40:03 5 [Warning] Aborted connection 5 to db: 'unconnected' user: 'unauthenticated' host: '172.19.0.3' (This connection closed normally without authentication) +bookstack-db | 2026-09-06 11:40:25 17 [Warning] Aborted connection 17 to db: 'unconnected' user: 'unauthenticated' host: '172.19.0.3' (This connection closed normally without authentication) diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/to-states.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/to-states.json new file mode 100644 index 00000000..b910bf46 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/to-states.json @@ -0,0 +1,14 @@ +{ + "bookstack": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + }, + "bookstack-db": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + } +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/verdict.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/verdict.json new file mode 100644 index 00000000..3d2de14a --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3a/verdict.json @@ -0,0 +1,25 @@ +{ + "harness_version": 1, + "edge": "E3a", + "app": "bookstack", + "note": "AUTHORED step (the catalog never carried it): app half alone", + "from": { + "bookstack": "lscr.io/linuxserver/bookstack:25.02.2", + "bookstack-db": "mariadb:11.6" + }, + "to": { + "bookstack": "lscr.io/linuxserver/bookstack:26.05.2", + "bookstack-db": "mariadb:11.6" + }, + "verdict": "proven", + "seed_read_before": true, + "seed_read_after": true, + "healthy_after": true, + "migration_observed": "bookstack | [migrations] started", + "abort": "starts-and-serves", + "abort_detail": null, + "duration_s": 15.7, + "measured_at": "2026-09-06T09:40:57Z", + "evidence": "evidence/E3a", + "total_s": 71.8 +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/abort-states.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/abort-states.json new file mode 100644 index 00000000..b910bf46 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/abort-states.json @@ -0,0 +1,14 @@ +{ + "bookstack": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + }, + "bookstack-db": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + } +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/compose-final.log b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/compose-final.log new file mode 100644 index 00000000..c92d94ed --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/compose-final.log @@ -0,0 +1,183 @@ +bookstack-db | 2026-09-06 11:41:44+02:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.6.2+maria~ubu2404 started. +bookstack-db | 2026-09-06 11:41:44+02:00 [Warn] [Entrypoint]: /sys/fs/cgroup///memory.pressure not writable, functionality unavailable to MariaDB +bookstack-db | 2026-09-06 11:41:44+02:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql' +bookstack-db | 2026-09-06 11:41:44+02:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.6.2+maria~ubu2404 started. +bookstack-db | 2026-09-06 11:41:44+02:00 [Note] [Entrypoint]: MariaDB upgrade not required +bookstack-db | 2026-09-06 11:41:44 0 [Note] Starting MariaDB 11.6.2-MariaDB-ubu2404 source revision d8dad8c3b54cd09fefce7bc3b9749f427eed9709 server_uid uxGayY7vyRLNrkVhQBGWZMg7sws= as process 1 +bookstack-db | 2026-09-06 11:41:44 0 [Note] InnoDB: Compressed tables use zlib 1.3 +bookstack-db | 2026-09-06 11:41:44 0 [Note] InnoDB: Number of transaction pools: 1 +bookstack-db | 2026-09-06 11:41:44 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions +bookstack-db | 2026-09-06 11:41:44 0 [Warning] mariadbd: io_uring_queue_init() failed with errno 0 +bookstack | [migrations] started +bookstack-db | 2026-09-06 11:41:44 0 [Warning] InnoDB: liburing disabled: falling back to innodb_use_native_aio=OFF +bookstack-db | 2026-09-06 11:41:44 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB +bookstack | [migrations] 01-nginx-site-confs-default: executing... +bookstack | [migrations] 01-nginx-site-confs-default: succeeded +bookstack | [migrations] 02-default-location: executing... +bookstack | [migrations] 02-default-location: succeeded +bookstack | [migrations] done +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | ██╗ ███████╗██╗ ██████╗ +bookstack-db | 2026-09-06 11:41:44 0 [Note] InnoDB: Completed initialization of buffer pool +bookstack | ██║ ██╔════╝██║██╔═══██╗ +bookstack-db | 2026-09-06 11:41:44 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes) +bookstack-db | 2026-09-06 11:41:44 0 [Note] InnoDB: End of log at LSN=2003287 +bookstack-db | 2026-09-06 11:41:44 0 [Note] InnoDB: Opened 3 undo tablespaces +bookstack-db | 2026-09-06 11:41:44 0 [Note] InnoDB: 128 rollback segments in 3 undo tablespaces are active. +bookstack-db | 2026-09-06 11:41:44 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ... +bookstack-db | 2026-09-06 11:41:44 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB. +bookstack-db | 2026-09-06 11:41:44 0 [Note] InnoDB: log sequence number 2003287; transaction id 2873 +bookstack-db | 2026-09-06 11:41:44 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool +bookstack-db | 2026-09-06 11:41:44 0 [Note] Plugin 'FEEDBACK' is disabled. +bookstack-db | 2026-09-06 11:41:44 0 [Note] Plugin 'wsrep-provider' is disabled. +bookstack-db | 2026-09-06 11:41:44 0 [Note] InnoDB: Buffer pool(s) load completed at 260906 11:41:44 +bookstack-db | 2026-09-06 11:41:48 0 [Note] Server socket created on IP: '0.0.0.0'. +bookstack-db | 2026-09-06 11:41:48 0 [Note] Server socket created on IP: '::'. +bookstack-db | 2026-09-06 11:41:48 0 [Note] mariadbd: Event Scheduler: Loaded 0 events +bookstack-db | 2026-09-06 11:41:48 0 [Note] mariadbd: ready for connections. +bookstack-db | Version: '11.6.2-MariaDB-ubu2404' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution +bookstack | ██║ ███████╗██║██║ ██║ +bookstack | ██║ ╚════██║██║██║ ██║ +bookstack | ███████╗███████║██║╚██████╔╝ +bookstack | ╚══════╝╚══════╝╚═╝ ╚═════╝ +bookstack | +bookstack | Brought to you by linuxserver.io +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | To support the app dev(s) visit: +bookstack | Bookstack: https://www.bookstackapp.com/donate/ +bookstack | +bookstack | To support LSIO projects visit: +bookstack | https://www.linuxserver.io/donate/ +bookstack | +bookstack | ─────────────────────────────────────── +bookstack | GID/UID +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | User UID: 1000 +bookstack | User GID: 1000 +bookstack | ─────────────────────────────────────── +bookstack | Linuxserver.io version: v26.05.2-ls276 +bookstack | Build-date: 2026-07-27T19:41:43+00:00 +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | Setting resolver to 127.0.0.11 +bookstack | Setting worker_processes to 4 +bookstack | generating self-signed keys in /config/keys, you can replace these with your own keys if required +bookstack | .+.........+.......+.....+.+...+..............+++++++++++++++++++++++++++++++++++++++*....+++++++++++++++++++++++++++++++++++++++*.+..+.........+....+.....+....+...............+..+...+.+.....+.+......++++++ +bookstack | ....+........+...+......+.+..+.+......+.....+......+....+........+.......+...+............+.....+......+...+.+.....+......+....+...........+...+.........+++++++++++++++++++++++++++++++++++++++*............+......+...+.....+......+.....................+.+........+....+.....+.......+.....+.........+.......+.....+....+..............+.+........+...+...+.+.....+.+..+.+...........+++++++++++++++++++++++++++++++++++++++*......+.+.....+.+...+..+.............+..+......+............+..................+.......+......+......+.....+....+.....+......+...+.+.....+.......+........+.+......+............+..+......+....+...+.....++++++ +bookstack | ----- +bookstack | Waiting for DB to be available +bookstack | +bookstack | INFO Preparing database. +bookstack | +bookstack | Creating migration table ...................................... 23.30ms DONE +bookstack | +bookstack | INFO Running migrations. +bookstack | +bookstack | 2014_10_12_000000_create_users_table ......................... 281.93ms DONE +bookstack | 2014_10_12_100000_create_password_resets_table ................ 67.95ms DONE +bookstack | 2015_07_12_114933_create_books_table .......................... 14.59ms DONE +bookstack | 2015_07_12_190027_create_pages_table .......................... 15.32ms DONE +bookstack | 2015_07_13_172121_create_images_table ......................... 13.99ms DONE +bookstack | 2015_07_27_172342_create_chapters_table ....................... 15.34ms DONE +bookstack | 2015_08_08_200447_add_users_to_entities ...................... 156.25ms DONE +bookstack | 2015_08_09_093534_create_page_revisions_table ................. 17.70ms DONE +bookstack | 2015_08_16_142133_create_activities_table ..................... 11.81ms DONE +bookstack | 2015_08_29_105422_add_roles_and_permissions .................. 325.57ms DONE +bookstack | 2015_08_30_125859_create_settings_table ....................... 12.58ms DONE +bookstack | 2015_08_31_175240_add_search_indexes ........................... 0.07ms DONE +bookstack | 2015_09_04_165821_create_social_accounts_table ................ 62.88ms DONE +bookstack | 2015_09_05_164707_add_email_confirmation_table ................ 79.14ms DONE +bookstack | 2015_11_21_145609_create_views_table .......................... 14.90ms DONE +bookstack | 2015_11_26_221857_add_entity_indexes ......................... 498.52ms DONE +bookstack | 2015_12_05_145049_fulltext_weighting ........................... 0.06ms DONE +bookstack | 2015_12_07_195238_add_image_upload_types ...................... 75.75ms DONE +bookstack | 2015_12_09_195748_add_user_avatars ............................ 19.55ms DONE +bookstack | 2016_01_11_210908_add_external_auth_to_users .................. 48.29ms DONE +bookstack | 2016_02_25_184030_add_slug_to_revisions ....................... 89.51ms DONE +bookstack | 2016_02_27_120329_update_permissions_and_roles ............... 168.57ms DONE +bookstack | 2016_02_28_084200_add_entity_access_controls ................. 268.80ms DONE +bookstack | 2016_03_09_203143_add_page_revision_types ..................... 42.57ms DONE +bookstack | 2016_03_13_082138_add_page_drafts ............................. 46.34ms DONE +bookstack | 2016_03_25_123157_add_markdown_support ........................ 37.35ms DONE +bookstack | 2016_04_09_100730_add_view_permissions_to_roles ............... 40.82ms DONE +bookstack | 2016_04_20_192649_create_joint_permissions_table ............. 302.48ms DONE +bookstack | 2016_05_06_185215_create_tags_table .......................... 112.20ms DONE +bookstack | 2016_07_07_181521_add_summary_to_page_revisions ............... 18.62ms DONE +bookstack | 2016_09_29_101449_remove_hidden_roles ......................... 74.62ms DONE +bookstack | 2016_10_09_142037_create_attachments_table .................... 66.52ms DONE +bookstack | 2017_01_21_163556_create_cache_table .......................... 41.57ms DONE +bookstack | 2017_01_21_163602_create_sessions_table ....................... 41.62ms DONE +bookstack | 2017_03_19_091553_create_search_index_table .................. 117.01ms DONE +bookstack | 2017_04_20_185112_add_revision_counts ......................... 64.95ms DONE +bookstack | 2017_07_02_152834_update_db_encoding_to_ut8mb4 ................. 0.07ms DONE +bookstack | 2017_08_01_130541_create_comments_table ....................... 86.73ms DONE +bookstack | 2017_08_29_102650_add_cover_image_display ..................... 21.13ms DONE +bookstack | 2018_07_15_173514_add_role_external_auth_id ................... 47.09ms DONE +bookstack | 2018_08_04_115700_create_bookshelves_table ................... 389.10ms DONE +bookstack | 2019_07_07_112515_add_template_support ........................ 48.36ms DONE +bookstack | 2019_08_17_140214_add_user_invites_table ...................... 60.98ms DONE +bookstack | 2019_12_29_120917_add_api_auth ................................ 90.78ms DONE +bookstack | 2020_08_04_111754_drop_joint_permissions_id .................. 109.84ms DONE +bookstack | 2020_08_04_131052_remove_role_name_field ...................... 18.62ms DONE +bookstack | 2020_09_19_094251_add_activity_indexes ........................ 49.30ms DONE +bookstack | 2020_09_27_210059_add_entity_soft_deletes ..................... 76.98ms DONE +bookstack | 2020_09_27_210528_create_deletions_table ...................... 92.18ms DONE +bookstack | 2020_11_07_232321_simplify_activities_table .................. 142.78ms DONE +bookstack | 2020_12_30_173528_add_owned_by_field_to_entities ............. 194.15ms DONE +bookstack | 2021_01_30_225441_add_settings_type_column .................... 17.75ms DONE +bookstack | 2021_03_08_215138_add_user_slug ............................... 51.19ms DONE +bookstack | 2021_05_15_173110_create_favourites_table ..................... 63.43ms DONE +bookstack | 2021_06_30_173111_create_mfa_values_table ..................... 64.09ms DONE +bookstack | 2021_07_03_085038_add_mfa_enforced_to_roles_table ............. 19.99ms DONE +bookstack | 2021_08_28_161743_add_export_role_permission ................... 4.67ms DONE +bookstack | 2021_09_26_044614_add_activities_ip_column .................... 20.63ms DONE +bookstack | 2021_11_26_070438_add_index_for_user_ip ....................... 27.19ms DONE +bookstack | 2021_12_07_111343_create_webhooks_table ...................... 126.26ms DONE +bookstack | 2021_12_13_152024_create_jobs_table ........................... 38.51ms DONE +bookstack | 2021_12_13_152120_create_failed_jobs_table .................... 38.97ms DONE +bookstack | 2022_01_03_154041_add_webhooks_timeout_error_columns .......... 73.69ms DONE +bookstack | 2022_04_17_101741_add_editor_change_field_and_permission ...... 24.94ms DONE +bookstack | 2022_04_25_140741_update_polymorphic_types .................... 15.16ms DONE +bookstack | 2022_07_16_170051_drop_joint_permission_type ................. 124.17ms DONE +bookstack | 2022_08_17_092941_create_references_table .................... 113.15ms DONE +bookstack | 2022_09_02_082910_fix_shelf_cover_image_types .................. 0.58ms DONE +bookstack | 2022_10_07_091406_flatten_entity_permissions_table ............ 88.86ms DONE +bookstack | 2022_10_08_104202_drop_entity_restricted_field ................ 93.09ms DONE +bookstack | 2023_01_24_104625_refactor_joint_permissions_storage ......... 142.44ms DONE +bookstack | 2023_01_28_141230_copy_color_settings_for_dark_mode ............ 0.99ms DONE +bookstack | 2023_02_20_093655_increase_attachments_path_length ............ 34.31ms DONE +bookstack | 2023_02_23_200227_add_updated_at_index_to_pages ............... 23.23ms DONE +bookstack | 2023_06_10_071823_remove_guest_user_secondary_roles ............ 1.81ms DONE +bookstack | 2023_06_25_181952_remove_bookshelf_create_entity_permissions ... 0.05ms DONE +bookstack | 2023_07_25_124945_add_receive_notifications_role_permissions ... 5.72ms DONE +bookstack | 2023_07_31_104430_create_watches_table ........................ 89.06ms DONE +bookstack | 2023_08_21_174248_increase_cache_size ......................... 28.33ms DONE +bookstack | 2023_12_02_104541_add_default_template_to_books ............... 23.82ms DONE +bookstack | 2023_12_17_140913_add_description_html_to_entities ............ 69.53ms DONE +bookstack | 2024_01_01_104542_add_default_template_to_chapters ............ 23.27ms DONE +bookstack | 2024_02_04_141358_add_views_updated_index ..................... 25.53ms DONE +bookstack | 2024_05_04_154409_rename_activity_relation_columns ............ 37.53ms DONE +bookstack | 2024_09_29_140340_ensure_editor_value_set ...................... 2.15ms DONE +bookstack | 2024_10_29_114420_add_import_role_permission ................... 6.62ms DONE +bookstack | 2024_11_02_160700_create_imports_table ........................ 38.10ms DONE +bookstack | 2024_11_27_171039_add_instance_id_setting ..................... 12.05ms DONE +bookstack | 2025_01_29_180933_create_sort_rules_table ..................... 15.04ms DONE +bookstack | 2025_02_05_150842_add_sort_rule_id_to_books ................... 22.84ms DONE +bookstack | 2025_04_18_215145_add_content_refs_and_archived_to_comments ... 64.90ms DONE +bookstack | 2025_09_02_111542_remove_unused_columns ....................... 65.37ms DONE +bookstack | 2025_09_15_132850_create_entities_table ...................... 264.90ms DONE +bookstack | 2025_09_15_134701_migrate_entity_data .......................... 9.04ms DONE +bookstack | 2025_09_15_134751_update_entity_relation_columns ............. 829.81ms DONE +bookstack | 2025_09_15_134813_drop_old_entity_tables ...................... 51.53ms DONE +bookstack | 2025_10_18_163331_clean_user_id_references ................... 470.16ms DONE +bookstack | 2025_10_22_134507_update_comments_relation_field_names ........ 38.65ms DONE +bookstack | 2025_11_23_161812_create_slug_history_table .................. 116.91ms DONE +bookstack | 2025_12_15_140219_create_mention_history_table ................ 65.12ms DONE +bookstack | 2025_12_19_103417_add_views_viewable_type_index ............... 23.60ms DONE +bookstack | 2026_04_19_141616_add_revision_view_all_permission ............. 7.79ms DONE +bookstack | +bookstack | [custom-init] No custom files found, skipping... +bookstack | [ls.io-init] done. diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/migration-lines.txt b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/migration-lines.txt new file mode 100644 index 00000000..a20781a7 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/migration-lines.txt @@ -0,0 +1 @@ +bookstack-db | 2026-09-06 11:41:37+02:00 [Note] [Entrypoint]: MariaDB upgrade (mariadb-upgrade or creating healthcheck users) required, but skipped due to $MARIADB_AUTO_UPGRADE setting \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/run.log b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/run.log new file mode 100644 index 00000000..c384becb --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/run.log @@ -0,0 +1,14 @@ +[09:41:01] E3b: deploying bookstack at FROM {'bookstack': 'lscr.io/linuxserver/bookstack:26.05.2', 'bookstack-db': 'mariadb:11.6'} +[09:41:34] FROM settled=True in 20.8s :: {"bookstack": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "bookstack-db": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}} +[09:41:35] bookstack: artisan create-admin rc=0 :: Admin account with email "spike-6b17d3aa@gate.invalid" successfully created! +[09:41:35] bookstack: readback of the seeded account found=True :: This will delete any configure multi-factor authentication methods for user: - ID: 3 - Name: spike-943f99 - Email: spike +[09:41:35] C1 (seed reads back BEFORE): True +[09:41:35] E3b: swapping to TO {'bookstack': 'lscr.io/linuxserver/bookstack:26.05.2', 'bookstack-db': 'mariadb:12.3'} +[09:41:42] TO up -d rc=0 +[09:41:42] TO settled=True in 0.2s :: {"bookstack": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "bookstack-db": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}} +[09:41:42] migration lines observed: 1 +[09:41:43] bookstack: readback of the seeded account found=True :: This will delete any configure multi-factor authentication methods for user: - ID: 3 - Name: spike-943f99 - Email: spike +[09:41:43] RESULT (seed reads back AFTER): True +[09:41:43] E3b: ABORT — putting the FROM images back +[09:41:50] bookstack: readback of the seeded account found=True :: This will delete any configure multi-factor authentication methods for user: - ID: 3 - Name: spike-943f99 - Email: spike +[09:41:50] ABORT: app came back in 0.2s; data present=True \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/to-full.log b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/to-full.log new file mode 100644 index 00000000..451b785e --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/to-full.log @@ -0,0 +1,184 @@ +bookstack | [migrations] started +bookstack | [migrations] 01-nginx-site-confs-default: executing... +bookstack | [migrations] 01-nginx-site-confs-default: succeeded +bookstack | [migrations] 02-default-location: executing... +bookstack | [migrations] 02-default-location: succeeded +bookstack | [migrations] done +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | ██╗ ███████╗██╗ ██████╗ +bookstack | ██║ ██╔════╝██║██╔═══██╗ +bookstack-db | 2026-09-06 11:41:36+02:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:12.3.3+maria~ubu2404 started. +bookstack-db | 2026-09-06 11:41:37+02:00 [Warn] [Entrypoint]: /sys/fs/cgroup///memory.pressure not writable, functionality unavailable to MariaDB +bookstack-db | 2026-09-06 11:41:37+02:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql' +bookstack-db | 2026-09-06 11:41:37+02:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:12.3.3+maria~ubu2404 started. +bookstack-db | 2026-09-06 11:41:37+02:00 [Note] [Entrypoint]: MariaDB upgrade (mariadb-upgrade or creating healthcheck users) required, but skipped due to $MARIADB_AUTO_UPGRADE setting +bookstack-db | 2026-09-06 11:41:37 0 [Note] Starting MariaDB 12.3.3-MariaDB-ubu2404 source revision 83e909fc2a0dbc394b4b683fb3fa2d7dcf26cc5e server_uid EeA58kUoZa1ZXw2cdj04/QXU0cI= as process 1 +bookstack-db | 2026-09-06 11:41:37 0 [Note] InnoDB: Compressed tables use zlib 1.3 +bookstack-db | 2026-09-06 11:41:37 0 [Note] InnoDB: Number of transaction pools: 1 +bookstack-db | 2026-09-06 11:41:37 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions +bookstack-db | 2026-09-06 11:41:37 0 [Warning] mariadbd: io_uring_queue_init() failed with EPERM: sysctl kernel.io_uring_disabled has the value 2, or 1 and the user of the process is not a member of sysctl kernel.io_uring_group. (see man 2 io_uring_setup). +bookstack | ██║ ███████╗██║██║ ██║ +bookstack-db | create_uring failed: falling back to libaio +bookstack | ██║ ╚════██║██║██║ ██║ +bookstack-db | 2026-09-06 11:41:37 0 [Note] InnoDB: Using Linux native AIO +bookstack | ███████╗███████║██║╚██████╔╝ +bookstack-db | 2026-09-06 11:41:37 0 [Note] InnoDB: innodb_buffer_pool_size_max=8388608m, innodb_buffer_pool_size=128m +bookstack | ╚══════╝╚══════╝╚═╝ ╚═════╝ +bookstack-db | 2026-09-06 11:41:37 0 [Note] InnoDB: Completed initialization of buffer pool +bookstack-db | 2026-09-06 11:41:37 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes) +bookstack-db | 2026-09-06 11:41:37 0 [Note] InnoDB: End of log at LSN=2003287 +bookstack-db | 2026-09-06 11:41:37 0 [Note] InnoDB: Opened 3 undo tablespaces +bookstack-db | 2026-09-06 11:41:37 0 [Note] InnoDB: 128 rollback segments in 3 undo tablespaces are active. +bookstack-db | 2026-09-06 11:41:37 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ... +bookstack-db | 2026-09-06 11:41:37 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB. +bookstack-db | 2026-09-06 11:41:37 0 [Note] InnoDB: log sequence number 2003287; transaction id 2873 +bookstack-db | 2026-09-06 11:41:37 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool +bookstack-db | 2026-09-06 11:41:37 0 [Note] Plugin 'FEEDBACK' is disabled. +bookstack-db | 2026-09-06 11:41:37 0 [Note] Plugin 'wsrep-provider' is disabled. +bookstack-db | 2026-09-06 11:41:37 0 [Note] InnoDB: Buffer pool(s) load completed at 260906 11:41:37 +bookstack-db | 2026-09-06 11:41:39 0 [Note] Server socket created on IP: '0.0.0.0', port: '3306'. +bookstack-db | 2026-09-06 11:41:39 0 [Note] Server socket created on IP: '::', port: '3306'. +bookstack-db | 2026-09-06 11:41:39 0 [Note] mariadbd: Event Scheduler: Loaded 0 events +bookstack-db | 2026-09-06 11:41:39 0 [Note] mariadbd: ready for connections. +bookstack-db | Version: '12.3.3-MariaDB-ubu2404' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution +bookstack | +bookstack | Brought to you by linuxserver.io +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | To support the app dev(s) visit: +bookstack | Bookstack: https://www.bookstackapp.com/donate/ +bookstack | +bookstack | To support LSIO projects visit: +bookstack | https://www.linuxserver.io/donate/ +bookstack | +bookstack | ─────────────────────────────────────── +bookstack | GID/UID +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | User UID: 1000 +bookstack | User GID: 1000 +bookstack | ─────────────────────────────────────── +bookstack | Linuxserver.io version: v26.05.2-ls276 +bookstack | Build-date: 2026-07-27T19:41:43+00:00 +bookstack | ─────────────────────────────────────── +bookstack | +bookstack | Setting resolver to 127.0.0.11 +bookstack | Setting worker_processes to 4 +bookstack | generating self-signed keys in /config/keys, you can replace these with your own keys if required +bookstack | .+.........+.......+.....+.+...+..............+++++++++++++++++++++++++++++++++++++++*....+++++++++++++++++++++++++++++++++++++++*.+..+.........+....+.....+....+...............+..+...+.+.....+.+......++++++ +bookstack | ....+........+...+......+.+..+.+......+.....+......+....+........+.......+...+............+.....+......+...+.+.....+......+....+...........+...+.........+++++++++++++++++++++++++++++++++++++++*............+......+...+.....+......+.....................+.+........+....+.....+.......+.....+.........+.......+.....+....+..............+.+........+...+...+.+.....+.+..+.+...........+++++++++++++++++++++++++++++++++++++++*......+.+.....+.+...+..+.............+..+......+............+..................+.......+......+......+.....+....+.....+......+...+.+.....+.......+........+.+......+............+..+......+....+...+.....++++++ +bookstack | ----- +bookstack | Waiting for DB to be available +bookstack | +bookstack | INFO Preparing database. +bookstack | +bookstack | Creating migration table ...................................... 23.30ms DONE +bookstack | +bookstack | INFO Running migrations. +bookstack | +bookstack | 2014_10_12_000000_create_users_table ......................... 281.93ms DONE +bookstack | 2014_10_12_100000_create_password_resets_table ................ 67.95ms DONE +bookstack | 2015_07_12_114933_create_books_table .......................... 14.59ms DONE +bookstack | 2015_07_12_190027_create_pages_table .......................... 15.32ms DONE +bookstack | 2015_07_13_172121_create_images_table ......................... 13.99ms DONE +bookstack | 2015_07_27_172342_create_chapters_table ....................... 15.34ms DONE +bookstack | 2015_08_08_200447_add_users_to_entities ...................... 156.25ms DONE +bookstack | 2015_08_09_093534_create_page_revisions_table ................. 17.70ms DONE +bookstack | 2015_08_16_142133_create_activities_table ..................... 11.81ms DONE +bookstack | 2015_08_29_105422_add_roles_and_permissions .................. 325.57ms DONE +bookstack | 2015_08_30_125859_create_settings_table ....................... 12.58ms DONE +bookstack | 2015_08_31_175240_add_search_indexes ........................... 0.07ms DONE +bookstack | 2015_09_04_165821_create_social_accounts_table ................ 62.88ms DONE +bookstack | 2015_09_05_164707_add_email_confirmation_table ................ 79.14ms DONE +bookstack | 2015_11_21_145609_create_views_table .......................... 14.90ms DONE +bookstack | 2015_11_26_221857_add_entity_indexes ......................... 498.52ms DONE +bookstack | 2015_12_05_145049_fulltext_weighting ........................... 0.06ms DONE +bookstack | 2015_12_07_195238_add_image_upload_types ...................... 75.75ms DONE +bookstack | 2015_12_09_195748_add_user_avatars ............................ 19.55ms DONE +bookstack | 2016_01_11_210908_add_external_auth_to_users .................. 48.29ms DONE +bookstack | 2016_02_25_184030_add_slug_to_revisions ....................... 89.51ms DONE +bookstack | 2016_02_27_120329_update_permissions_and_roles ............... 168.57ms DONE +bookstack | 2016_02_28_084200_add_entity_access_controls ................. 268.80ms DONE +bookstack | 2016_03_09_203143_add_page_revision_types ..................... 42.57ms DONE +bookstack | 2016_03_13_082138_add_page_drafts ............................. 46.34ms DONE +bookstack | 2016_03_25_123157_add_markdown_support ........................ 37.35ms DONE +bookstack | 2016_04_09_100730_add_view_permissions_to_roles ............... 40.82ms DONE +bookstack | 2016_04_20_192649_create_joint_permissions_table ............. 302.48ms DONE +bookstack | 2016_05_06_185215_create_tags_table .......................... 112.20ms DONE +bookstack | 2016_07_07_181521_add_summary_to_page_revisions ............... 18.62ms DONE +bookstack | 2016_09_29_101449_remove_hidden_roles ......................... 74.62ms DONE +bookstack | 2016_10_09_142037_create_attachments_table .................... 66.52ms DONE +bookstack | 2017_01_21_163556_create_cache_table .......................... 41.57ms DONE +bookstack | 2017_01_21_163602_create_sessions_table ....................... 41.62ms DONE +bookstack | 2017_03_19_091553_create_search_index_table .................. 117.01ms DONE +bookstack | 2017_04_20_185112_add_revision_counts ......................... 64.95ms DONE +bookstack | 2017_07_02_152834_update_db_encoding_to_ut8mb4 ................. 0.07ms DONE +bookstack | 2017_08_01_130541_create_comments_table ....................... 86.73ms DONE +bookstack | 2017_08_29_102650_add_cover_image_display ..................... 21.13ms DONE +bookstack | 2018_07_15_173514_add_role_external_auth_id ................... 47.09ms DONE +bookstack | 2018_08_04_115700_create_bookshelves_table ................... 389.10ms DONE +bookstack | 2019_07_07_112515_add_template_support ........................ 48.36ms DONE +bookstack | 2019_08_17_140214_add_user_invites_table ...................... 60.98ms DONE +bookstack | 2019_12_29_120917_add_api_auth ................................ 90.78ms DONE +bookstack | 2020_08_04_111754_drop_joint_permissions_id .................. 109.84ms DONE +bookstack | 2020_08_04_131052_remove_role_name_field ...................... 18.62ms DONE +bookstack | 2020_09_19_094251_add_activity_indexes ........................ 49.30ms DONE +bookstack | 2020_09_27_210059_add_entity_soft_deletes ..................... 76.98ms DONE +bookstack | 2020_09_27_210528_create_deletions_table ...................... 92.18ms DONE +bookstack | 2020_11_07_232321_simplify_activities_table .................. 142.78ms DONE +bookstack | 2020_12_30_173528_add_owned_by_field_to_entities ............. 194.15ms DONE +bookstack | 2021_01_30_225441_add_settings_type_column .................... 17.75ms DONE +bookstack | 2021_03_08_215138_add_user_slug ............................... 51.19ms DONE +bookstack | 2021_05_15_173110_create_favourites_table ..................... 63.43ms DONE +bookstack | 2021_06_30_173111_create_mfa_values_table ..................... 64.09ms DONE +bookstack | 2021_07_03_085038_add_mfa_enforced_to_roles_table ............. 19.99ms DONE +bookstack | 2021_08_28_161743_add_export_role_permission ................... 4.67ms DONE +bookstack | 2021_09_26_044614_add_activities_ip_column .................... 20.63ms DONE +bookstack | 2021_11_26_070438_add_index_for_user_ip ....................... 27.19ms DONE +bookstack | 2021_12_07_111343_create_webhooks_table ...................... 126.26ms DONE +bookstack | 2021_12_13_152024_create_jobs_table ........................... 38.51ms DONE +bookstack | 2021_12_13_152120_create_failed_jobs_table .................... 38.97ms DONE +bookstack | 2022_01_03_154041_add_webhooks_timeout_error_columns .......... 73.69ms DONE +bookstack | 2022_04_17_101741_add_editor_change_field_and_permission ...... 24.94ms DONE +bookstack | 2022_04_25_140741_update_polymorphic_types .................... 15.16ms DONE +bookstack | 2022_07_16_170051_drop_joint_permission_type ................. 124.17ms DONE +bookstack | 2022_08_17_092941_create_references_table .................... 113.15ms DONE +bookstack | 2022_09_02_082910_fix_shelf_cover_image_types .................. 0.58ms DONE +bookstack | 2022_10_07_091406_flatten_entity_permissions_table ............ 88.86ms DONE +bookstack | 2022_10_08_104202_drop_entity_restricted_field ................ 93.09ms DONE +bookstack | 2023_01_24_104625_refactor_joint_permissions_storage ......... 142.44ms DONE +bookstack | 2023_01_28_141230_copy_color_settings_for_dark_mode ............ 0.99ms DONE +bookstack | 2023_02_20_093655_increase_attachments_path_length ............ 34.31ms DONE +bookstack | 2023_02_23_200227_add_updated_at_index_to_pages ............... 23.23ms DONE +bookstack | 2023_06_10_071823_remove_guest_user_secondary_roles ............ 1.81ms DONE +bookstack | 2023_06_25_181952_remove_bookshelf_create_entity_permissions ... 0.05ms DONE +bookstack | 2023_07_25_124945_add_receive_notifications_role_permissions ... 5.72ms DONE +bookstack | 2023_07_31_104430_create_watches_table ........................ 89.06ms DONE +bookstack | 2023_08_21_174248_increase_cache_size ......................... 28.33ms DONE +bookstack | 2023_12_02_104541_add_default_template_to_books ............... 23.82ms DONE +bookstack | 2023_12_17_140913_add_description_html_to_entities ............ 69.53ms DONE +bookstack | 2024_01_01_104542_add_default_template_to_chapters ............ 23.27ms DONE +bookstack | 2024_02_04_141358_add_views_updated_index ..................... 25.53ms DONE +bookstack | 2024_05_04_154409_rename_activity_relation_columns ............ 37.53ms DONE +bookstack | 2024_09_29_140340_ensure_editor_value_set ...................... 2.15ms DONE +bookstack | 2024_10_29_114420_add_import_role_permission ................... 6.62ms DONE +bookstack | 2024_11_02_160700_create_imports_table ........................ 38.10ms DONE +bookstack | 2024_11_27_171039_add_instance_id_setting ..................... 12.05ms DONE +bookstack | 2025_01_29_180933_create_sort_rules_table ..................... 15.04ms DONE +bookstack | 2025_02_05_150842_add_sort_rule_id_to_books ................... 22.84ms DONE +bookstack | 2025_04_18_215145_add_content_refs_and_archived_to_comments ... 64.90ms DONE +bookstack | 2025_09_02_111542_remove_unused_columns ....................... 65.37ms DONE +bookstack | 2025_09_15_132850_create_entities_table ...................... 264.90ms DONE +bookstack | 2025_09_15_134701_migrate_entity_data .......................... 9.04ms DONE +bookstack | 2025_09_15_134751_update_entity_relation_columns ............. 829.81ms DONE +bookstack | 2025_09_15_134813_drop_old_entity_tables ...................... 51.53ms DONE +bookstack | 2025_10_18_163331_clean_user_id_references ................... 470.16ms DONE +bookstack | 2025_10_22_134507_update_comments_relation_field_names ........ 38.65ms DONE +bookstack | 2025_11_23_161812_create_slug_history_table .................. 116.91ms DONE +bookstack | 2025_12_15_140219_create_mention_history_table ................ 65.12ms DONE +bookstack | 2025_12_19_103417_add_views_viewable_type_index ............... 23.60ms DONE +bookstack | 2026_04_19_141616_add_revision_view_all_permission ............. 7.79ms DONE +bookstack | +bookstack | [custom-init] No custom files found, skipping... +bookstack | [ls.io-init] done. diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/to-states.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/to-states.json new file mode 100644 index 00000000..b910bf46 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/to-states.json @@ -0,0 +1,14 @@ +{ + "bookstack": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + }, + "bookstack-db": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + } +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/verdict.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/verdict.json new file mode 100644 index 00000000..bfb94ac2 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/E3b/verdict.json @@ -0,0 +1,25 @@ +{ + "harness_version": 1, + "edge": "E3b", + "app": "bookstack", + "note": "AUTHORED step: engine half alone", + "from": { + "bookstack": "lscr.io/linuxserver/bookstack:26.05.2", + "bookstack-db": "mariadb:11.6" + }, + "to": { + "bookstack": "lscr.io/linuxserver/bookstack:26.05.2", + "bookstack-db": "mariadb:12.3" + }, + "verdict": "proven", + "seed_read_before": true, + "seed_read_after": true, + "healthy_after": true, + "migration_observed": "bookstack-db | 2026-09-06 11:41:37+02:00 [Note] [Entrypoint]: MariaDB upgrade (mariadb-upgrade or creating healthcheck users) required, but skipped due to $MARIADB_AUTO_UPGRADE setting", + "abort": "starts-and-serves", + "abort_detail": null, + "duration_s": 0.2, + "measured_at": "2026-09-06T09:41:50Z", + "evidence": "evidence/E3b", + "total_s": 48.8 +} \ No newline at end of file diff --git a/documentation/audits/upgrade-spike-2026-09-06/evidence/summary.json b/documentation/audits/upgrade-spike-2026-09-06/evidence/summary.json new file mode 100644 index 00000000..7b1ce393 --- /dev/null +++ b/documentation/audits/upgrade-spike-2026-09-06/evidence/summary.json @@ -0,0 +1,77 @@ +[ + { + "harness_version": 1, + "edge": "E3", + "app": "bookstack", + "note": "catalog transition 0b73e5e: app AND engine together", + "from": { + "bookstack": "lscr.io/linuxserver/bookstack:25.02.2", + "bookstack-db": "mariadb:11.6" + }, + "to": { + "bookstack": "lscr.io/linuxserver/bookstack:26.05.2", + "bookstack-db": "mariadb:12.3" + }, + "verdict": "proven", + "seed_read_before": true, + "seed_read_after": true, + "healthy_after": true, + "migration_observed": "bookstack-db | 2026-09-06 11:38:57+02:00 [Note] [Entrypoint]: MariaDB upgrade (mariadb-upgrade or creating healthcheck users) required, but skipped due to $MARIADB_AUTO_UPGRADE setting", + "abort": "starts-and-serves", + "abort_detail": null, + "duration_s": 15.7, + "measured_at": "2026-09-06T09:39:41Z", + "evidence": "evidence/E3", + "total_s": 77.5 + }, + { + "harness_version": 1, + "edge": "E3a", + "app": "bookstack", + "note": "AUTHORED step (the catalog never carried it): app half alone", + "from": { + "bookstack": "lscr.io/linuxserver/bookstack:25.02.2", + "bookstack-db": "mariadb:11.6" + }, + "to": { + "bookstack": "lscr.io/linuxserver/bookstack:26.05.2", + "bookstack-db": "mariadb:11.6" + }, + "verdict": "proven", + "seed_read_before": true, + "seed_read_after": true, + "healthy_after": true, + "migration_observed": "bookstack | [migrations] started", + "abort": "starts-and-serves", + "abort_detail": null, + "duration_s": 15.7, + "measured_at": "2026-09-06T09:40:57Z", + "evidence": "evidence/E3a", + "total_s": 71.8 + }, + { + "harness_version": 1, + "edge": "E3b", + "app": "bookstack", + "note": "AUTHORED step: engine half alone", + "from": { + "bookstack": "lscr.io/linuxserver/bookstack:26.05.2", + "bookstack-db": "mariadb:11.6" + }, + "to": { + "bookstack": "lscr.io/linuxserver/bookstack:26.05.2", + "bookstack-db": "mariadb:12.3" + }, + "verdict": "proven", + "seed_read_before": true, + "seed_read_after": true, + "healthy_after": true, + "migration_observed": "bookstack-db | 2026-09-06 11:41:37+02:00 [Note] [Entrypoint]: MariaDB upgrade (mariadb-upgrade or creating healthcheck users) required, but skipped due to $MARIADB_AUTO_UPGRADE setting", + "abort": "starts-and-serves", + "abort_detail": null, + "duration_s": 0.2, + "measured_at": "2026-09-06T09:41:50Z", + "evidence": "evidence/E3b", + "total_s": 48.8 + } +] \ No newline at end of file diff --git a/documentation/backlog/CLOSED-ITEMS.md b/documentation/backlog/CLOSED-ITEMS.md index a5ed257a..0b455638 100644 --- a/documentation/backlog/CLOSED-ITEMS.md +++ b/documentation/backlog/CLOSED-ITEMS.md @@ -26,6 +26,7 @@ --- +| **R-449** | **UPDATE ARC SLICE 5 — an upgrade test that runs again. BUILT AND RUN 2026-09-06:** `app-catalog-felhom.eu/scripts/upgrade-test.py` + `upgrade_fixtures.py`, 7 edges across 3 apps, evidence in `audits/upgrade-spike-2026-09-06/`. **Reasoning kept — success is an APPLICATION-LEVEL READBACK, never file identity:** `survive2.py`'s sha256+inode rule is right for a redeploy and WRONG for an upgrade, because a migration is supposed to rewrite files and that rule would fail every correct upgrade. **Reasoning kept — nothing is ever seeded into a volume by hand (R-156);** an app with no non-browser route is recorded `inconclusive`, which is a result and not a licence to plant a file. **Reasoning kept — run the negative control FIRST:** C3's TO image exits immediately and came back `failed`; a harness that cannot fail a known-broken upgrade proves nothing with its greens. **WHAT IT MEASURED:** all five real catalog upgrades kept the customer's data; and **whether an upgrade can be undone is a property of the individual APP, not of upgrades** — docmost REFUSES (*"corrupted migrations: previously executed migration 20260213T085259-notifications is missing"*), privatebin does not, which independently reproduces the Nextcloud finding on a second app by a DIFFERENT mechanism and puts two measurements behind §4's ruling that "rollback" is the wrong word. **It also found a defect in our own catalog (R-459).** **What stays open, as its own rows rather than inside this one:** R-459 (the skipped MariaDB datadir upgrade), R-460 (bookstack's file half is unprovable headlessly), R-462 (the widening, costed). Full original text: `git show 417df06f3529:documentation/backlog/OPEN-ITEMS.md`. | **CLOSED 2026-09-06 — harness built, run, and proven by a red negative control** | `audits/SPIKE-upgrade-test-2026-09-06.md`; `audits/upgrade-spike-2026-09-06/evidence/`; catalog `0474ce387e6f` | | **R-438** | **The catalog sync rewrote a DEPLOYED app's `docker-compose.yml` and no architecture document recorded that it did. BOTH HALVES NOW DISCHARGED — the document was written 2026-09-02, the behaviour was changed in controller v0.235.0 (2026-09-06).** `Syncer.copyTemplates` copied into every stack folder on a 15-minute cycle with **no deployed check**, so a deployed app's file and its running containers disagreed from that moment, and the next `compose up -d` from any of thirteen call sites resolved the disagreement by upgrading — measured live: the sync rewrote the file at 17:45:17Z while the container went on running the old image, a restart then upgraded it in 18.3 s **with a pull**, and a boot reconciliation upgraded it **with nobody pressing anything**. **Reasoning kept — the distinction this row existed to protect:** `RestartStack`'s use of `up -d` to pick up template changes was **CHOSEN and written down in its own comment**, so reversing it was an operator DECISION, not a bug fix; that is why the row stayed open through v0.233.0 and v0.234.0 while only the documentation half was done. **Reasoning kept — one fear was measured SMALLER than stated:** a plain power cut does NOT upgrade anything, because Docker's `restart: unless-stopped` restores the containers on the old image and the reconciler logs `no boot-orphaned apps`; the unattended upgrade needs the narrower precondition *"and the app did not come back"*. Full original text: `git show bc47dd4ef997:documentation/backlog/OPEN-ITEMS.md`. | **CLOSED 2026-09-06 — documented 2026-09-02, behaviour changed in controller v0.235.0** | `audits/SPIKE-app-update-2026-09-01.md` §2, §3, §8; `architecture/09-update-architecture.md`; `tests/VALIDATION-update-slice3-2026-09-06.md` | | **R-447** | **UPDATE ARC SLICE 3 — the live compose file is now DERIVED; the syncer renders instead of copying. SHIPPED controller v0.235.0 (2026-09-06).** The pin lives in `app.yaml` (`pinned_images`), the definition it came from is stored beside the app as `applied-compose.yml`, and `Syncer.renderSource` writes the catalog template verbatim while the catalog still offers the pinned version and the stored definition once it moves past it. **Reasoning kept — the ruling, in the operator's own words:** *while the catalog is offering the same version you are running, its fixes flow to you; the moment it moves to a newer version, you are frozen at what you have until you choose to update.* **Reasoning kept — why nothing was added to the thirteen `compose up -d` call sites:** most of them are REPAIRS (the boot reconciler, the drive-return gate, the app-stop guard), and **a repair path that refuses to repair leaves a customer's app down, which is worse than the problem**; they were made safe by removing the reason, not by gating them. **Reasoning kept — why the frozen branch writes a WHOLE file and never a substitution:** `wger 2.6` needs a full DB configuration the older template cannot supply, so an old image under a new template is a third state nobody chose. **Reasoning kept — why this is not "skip deployed apps" (option B, rejected):** that also stops health-check fixes, memory limits and new deploy fields, and destroys the self-healing measured live in the spike §3. **Reasoning kept — `pinned_images` is INTENT and `installed_images` is an OBSERVATION; never feed one from the other** (the R-166 category error, one field over). Full original text: `git show bc47dd4ef997:documentation/backlog/OPEN-ITEMS.md`. | **CLOSED 2026-09-06 — SHIPPED controller v0.235.0** | `architecture/09-update-architecture.md` §3.4, §5; `tests/VALIDATION-update-slice3-2026-09-06.md`; controller `CHANGELOG.md` v0.235.0 | | **R-441** | **The restore path and the catalog sync disagreed about the image, and the SYNC WON within 15 minutes. CLOSED controller v0.235.0 (2026-09-06).** `stackAdapter.RecreateStackDefinitionFromUnit` wrote the recovery unit's captured `docker-compose.yml` — carrying the OLD pin — into the live stack dir, and `Syncer.copyIfChanged` overwrote it from the catalog on the next tick, so a restore's image-level recovery had a **<=15-minute half-life**. The restore now PINS to what the unit captured and stores it as the applied definition, so the render obeys the restored file instead. **Reasoning kept:** the overwrite half was MEASURED live 2026-09-01 (`[INFO] [sync] Updated bentopdf/docker-compose.yml` at 18:10:29Z over a locally-modified file); the "the restore writes to that same path" half was READ, and this closure rests on the render behaviour being measured live rather than on the reading. Full original text: `git show bc47dd4ef997:documentation/backlog/OPEN-ITEMS.md`. | **CLOSED 2026-09-06 — SHIPPED controller v0.235.0** | `internal/stacks/pin.go`; `TestGroupF_RestorePinIsReportedToTheSyncer`; `tests/VALIDATION-update-slice3-2026-09-06.md` | diff --git a/documentation/backlog/OPEN-ITEMS.md b/documentation/backlog/OPEN-ITEMS.md index 2673eab4..5abb359f 100644 --- a/documentation/backlog/OPEN-ITEMS.md +++ b/documentation/backlog/OPEN-ITEMS.md @@ -683,7 +683,6 @@ class (an image `VOLUME` at an unmounted path) is still live — `immich-server` | **R-445** | **[P3-LOW] Hub app telemetry survives the app's removal, so a 15-minute throwaway now sets a FLEET-WIDE memory recommendation.** MEASURED 2026-09-01: this spike's Phase 6 Nextcloud existed for ~15 minutes on demo-hp, spent part of it crash-looping, and was then removed with all volumes. The hub's `/apps/nextcloud` page still reports `Deployments`, `Avg Memory 208 MB`, `P95 Memory 280 MB` and **`Suggested Limit (P95x1.2) = 352 MB`**, plus three MariaDB `io_uring` rows under Known Issues attributed to demo-hp. **The suggested limit is an operator-facing recommendation derived from a sample that no longer exists anywhere** — and Nextcloud is a real catalog app whose limit someone may act on. **RETAINED DELIBERATELY BY THIS RUN, NOT CLEARED, and the reason is part of the row:** the hub offers `POST /apps/nextcloud/reset-telemetry` whose own confirm reads *"Delete all telemetry data for nextcloud? This cannot be undone."* — an irreversible write on the operator's surface, and the operator authorised Phase 6, not this. **The one-line command is recorded in the audit doc so it is a decision, not a task.** The general question is the row: should telemetry for an app with zero live deployments age out, or be excluded from the suggestion? Owner: **VIKTOR rules, CC implements.** `audits/SPIKE-app-update-2026-09-01.md` | **OPEN — rank P3-LOW; owner: VIKTOR rules, CC implements** | | **R-446** | **[P2-MEDIUM] „Naprakész" can be FALSE, and the badge that says it cannot tell.** Slice 2 (controller v0.233.0, 2026-09-02) compares the RECORDED image reference per compose service against the reference the current template pins, and **queries no registry** — deliberately: a customer's box must not depend on reaching eight upstream registries to render a page (`felhom-controller/controller/internal/web/updatebadge.go`, `compareInstalledToTemplate`). **For the 23 floating pins that comparison is blind by construction:** `postgres:16-alpine`, `mariadb:11.6` and 21 others can carry an identical reference over an image that has moved. **MEASURED, not theorised — spike §5 found `mariadb:11.4` and `mariadb:12.3` had BOTH already moved upstream while two fully-pinned CONTROLS held.** So `romm` and `bookstack` on demo-hp would read „Naprakész" over a database engine build that is not the one the catalog now resolves to. **This is a KNOWN LIMITATION OF A SHIPPED FEATURE, filed the same session rather than left implicit**, and it is stated in the same words in `architecture/09-update-architecture.md` §8.1 and in the controller's `README.md`. The close is a digest comparison against the registry, which needs a network call, a cache and a failure posture — it is not a one-liner and it is not slice 2's job. **Depends on R-440**, whose fix (stop floating) would remove the problem instead of measuring it — take that route first if it is available. `architecture/09-update-architecture.md` | **OPEN — rank P2-MEDIUM; owner: CC** | | **R-448** | **[P2-MEDIUM] UPDATE ARC SLICE 4 — a guarded update: a verified backup as a precondition, an abort path, and the truth at the moment of action.** Three parts, each already evidenced. (a) **The precondition is a VERIFIED RECENT BACKUP, not a new copy** (operator ruling 2026-09-02); the guest-snapshot alternative must be SPIKED before anything is designed around it. Today's safety machinery is DATABASE-ONLY (`Manager.writeSafetyDump`, `internal/backup/offbox_reconstitute.go:207`) and the file half was never priced (spike §6). (b) **The abort path, never a "rollback"** — spike §7 proved the word is wrong: once a migration has run, the old image refuses to start on the migrated data. The two available shapes are ABORT (before anything migrated) and RESTORE FROM A COPY (after). (c) **Truth at the moment of action** — this subsumes **R-443**: the Update button returns HTTP 200 over an app it has just broken and the alarm arrives 5m16s later. `architecture/09-update-architecture.md` §3, §4 | **READY — rank P2-MEDIUM; owner: CC proposes, VIKTOR rules on (c)** | -| **R-449** | **[P2-MEDIUM] UPDATE ARC SLICE 5 — an upgrade test that proves a real one-major upgrade end to end, INCLUDING the abort path.** Spike §7 performed the pieces by hand on Nextcloud (31.0.14 → 32.0.9 ran the migration; 31.0.14 → 34.0.1 was refused by the app and reported as SUCCESS by the product; the downgrade attempt was refused with the data intact). **None of it is a test that runs again.** A one-off measurement that nothing repeats decays into a claim — this project's most-repeated defect class. The test must assert the CONSEQUENCE (does the app serve after the upgrade? does the abort put it back?), not the mechanism. Venue: a Tier-0 box; `runbooks/target-selection.md` names which. `architecture/09-update-architecture.md` §6 | **READY — rank P2-MEDIUM; owner: CC** | | **R-450** | **[P2-MEDIUM] UPDATE ARC SLICE 6 — a version sequence: automatic WITHIN a major, never ACROSS one, and an engine change gets its OWN edge.** The first half is an operator ruling of 2026-09-02 and its justification is R-449's measurement: a cross-major jump can be refused by the app itself and cannot be undone. **The second half is a rule recorded now, while it is cheap:** an engine change must never be bundled with an app version bump. `bookstack`'s `0b73e5e` moved the application 25.02.2 → 26.05.2 **and** MariaDB 11.6 → 12.3 in one commit — **two migrations behind one edge**, and an unreadable failure when it breaks. Needs a catalog-side convention and, eventually, a gate. `architecture/09-update-architecture.md` §6 | **READY — rank P2-MEDIUM; owner: VIKTOR rules, CC implements** | | **R-451** | **[P3-LOW] UPDATE ARC SLICE 7 — a fleet sweep: the operator can SEE, and MOVE, how far behind every box is.** Slices 1 and 2 make one box's state visible on that box's own pages. The operator has no fleet view, and **it is not derivable from what is already reported: the hub's report payload carries container name, state, CPU and memory, and NO image field at all** (spike §5, which is why Peti's box could only be recorded UNKNOWN). So this is a hub-side change as well as a controller one. Rank LOW today because the fleet is two enrolled boxes; it rises with the fleet. `architecture/09-update-architecture.md` §6, §8.4 | **READY — rank P3-LOW; owner: CC** | | **R-452** | **[P3-LOW] Nothing enforces `catalog_since`, so the one number the update badge shows can silently under-report.** `app-catalog-felhom.eu` `CLAUDE.md` now states the rule — any commit that changes an `image:` line must set that app's `catalog_since` to the same day — and all 53 apps were backfilled from git history on 2026-09-02 (`69761cf`). **A rule with no instrument is a wish; that is this project's most-repeated finding and this row exists so it is not repeated silently.** A stale `catalog_since` makes „Frissítés elérhető — N napja" under-report N, which is the single number the badge exists to give. **WHY IT WAS NOT BUILT IN THE SAME SESSION, stated rather than implied:** the gate would have to diff an `image:` line against the PARENT commit, and `catalog_gates.py` runs under a runner that fetches at `--depth 1` — there is no parent to diff against. The gate therefore needs a deeper fetch, which is a change to the CI shape and not to a script. **This is the R-421 class in advance: an enumerated gap becomes a row in the same session it is enumerated.** `architecture/09-update-architecture.md` §8.2 | **READY — rank P3-LOW; owner: CC** | @@ -692,6 +691,10 @@ class (an image `VOLUME` at an unmounted path) is still live — `immich-server` | **R-456** | **[P3-LOW] A partly-dead stack is not a boot orphan, and that is written down nowhere.** MEASURED 2026-09-02 on demo-hp while validating v0.233.0: `docker rm -f bookstack` (leaving `bookstack-db` running) then a controller restart produced `Boot reconciliation: 1 boot-orphaned app(s) found: [bentopdf]` — **bookstack was NOT selected**, although the app container was gone and `desired_state: running` was recorded. Removing `bookstack-db` as well made the whole stack orphaned and the very next pass repaired it in 6.3 s. **So `bootrecon.isBootOrphan` requires the stack as a WHOLE to be down; one live member is enough to make it invisible to the reconciler.** **NOT called a defect, and the reason is part of the row:** `StateDegraded` IS in `IsDownState`, and the crash-loop/dead-app alarm path (`classifyRunStates`) does count a degraded stack as down — so the customer IS told; it is the automatic REPAIR that does not fire, and there may be a good reason (repairing half a stack while its DB is live is not obviously safe). **What is certain is that nobody has written the rule down**, so the next session re-derives it the same way this one did — by watching a reconciliation not happen, which is an absent observable and the weakest possible evidence. Either state the rule in `02-controller-module-map.md` with a test pinning it, or change it. Owner: **CC.** `tests/VALIDATION-update-slice12-2026-09-02.md` §2.2 | **READY — rank P3-LOW; owner: CC** | | **R-457** | **[P3-LOW] A test that hardcodes a date AND asserts an age derived from it is green on the day it is written and red the next morning — one instance PROVEN, six candidate files named.** MEASURED 2026-09-03: `TestGroupD_BadgeRendersOnBothSurfaces` (shipped the previous day in v0.233.0) pinned a fixture `catalog_since: "2026-07-18"` and asserted the rendered string `"Frissítés elérhető — 46 napja"`. **The pure badge tests inject a clock; the RENDER test does not and cannot** — it goes through the production templates, which call the funcmap entry `updateBadge`, which reads `time.Now()`. The suite was green on 2026-09-02 and **FAILED on 2026-09-03** with *"the behind badge is missing"* on both surfaces, because the true answer had become 47. **Fixed by DERIVING the fixture** — `catalog_since` is computed as *today minus 46 days*, so the test asserts the real number through the real clock and cannot rot. **THE CLASS, which is why this is a row and not just a fix:** a clock-reading test that also carries a date LITERAL is a bomb with a fuse of unknown length, and the suite being green is not evidence it is defused — it is evidence the fuse has not burned down yet. **NAMED AS UNCHECKED CANDIDATES, NOT ACCUSED** — six other test files contain both a `20xx-xx-xx` literal and `time.Now()`: `internal/backup/offbox_test.go`, `internal/web/handler_export_upload_test.go`, `internal/web/r103_tier2_action_test.go`, `internal/web/dashboard_backup_card_test.go`, `internal/web/async_restore_test.go`, `internal/stacks/installed_test.go`. Mixing the two is not itself a defect — it is one only where a literal feeds an assertion evaluated against the real clock — so each needs reading, which is a sweep and not this session. **The instrument that would end the class:** run the suite once under a faked future date in CI and see what turns red. Owner: **CC.** `felhom-controller` v0.234.0 CHANGELOG | **READY — rank P3-LOW; owner: CC** | | **R-458** | **[P3-LOW] `.felhom.yml` keeps flowing to an app whose compose file is FROZEN, so a frozen app can receive a health check written for a version it is not running.** The v0.235.0 render freezes `docker-compose.yml` for a pinned app once the catalog moves past its version, but copies `.felhom.yml` **verbatim in every case** (`Syncer.copyTemplates`). **The asymmetry is deliberate and both directions were considered:** `.felhom.yml` carries no image, and it carries `catalog_since` — the single input the update badge uses to say *„Frissítés elérhető — N napja"* — so freezing it would silently withhold the one number that tells a customer they are behind, i.e. it would break slice 2 to protect slice 3. **What it costs:** the file also carries the controller-side `healthcheck:` block and resource hints, so a template updated for a newer version can hand a frozen app a probe written for software it is not running. **THE FAILURE DIRECTION IS A FALSE ALARM, NEVER DATA LOSS** — the app keeps running; at worst it renders as degraded and, if it persisted, could reach the dead-app alarm path. That is the same class as R-330's false e-mails, which is why this is a row and not a footnote. **Not fixed now, and the reason is that the cheap fix is wrong:** freezing the whole file breaks the badge, and freezing only the `healthcheck:` key means the syncer would have to parse and re-assemble a customer-facing metadata file — new surface on the one path that touches every app on every box every 15 minutes. **What would settle it:** whether any catalog `healthcheck:` has ever been changed in the same commit as an `image:` line (measurable from the catalog's own history, no box needed). If the answer is "never", the exposure is theoretical and the row can be closed by measurement instead of by code. Owner: **CC.** `architecture/09-update-architecture.md` §5.4, §8.5 | **READY — rank P3-LOW; owner: CC** | +| **R-459** | **[P2-MEDIUM] Our own bookstack template moves MariaDB across a MAJOR while the image tells us, in its own words, that the datadir upgrade it needs is being SKIPPED.** MEASURED 2026-09-06 in a throwaway guest, on the catalog's own transition `0b73e5e` (mariadb `11.6` → `12.3`): the moment 12.3 starts on the 11.6 datadir it logs **`[Note] [Entrypoint]: MariaDB upgrade (mariadb-upgrade or creating healthcheck users) required, but skipped due to $MARIADB_AUTO_UPGRADE setting`** — and then serves normally. `templates/bookstack/docker-compose.yml` sets **no `MARIADB_*` environment at all**, so `MARIADB_AUTO_UPGRADE` is unset and the entrypoint declines to run `mariadb-upgrade`. **THE CAUSE IS ASSIGNED, NOT GUESSED:** the edge was decomposed, and the app half alone (`E3a`, engine held at 11.6) produces **no** upgrade line while both edges that move the engine (`E3`, `E3b`) produce it. A bundled edge could never have said which half. **AND IT EXPLAINS A RESULT THAT LOOKED LIKE GOOD NEWS:** the abort of E3 "worked" — 11.6 came back and served the data, logging `MariaDB upgrade not required` — **because the datadir was never converted.** The reversibility is a side-effect of an upgrade that did not fully happen. **WHAT IS NOT ESTABLISHED, and this row must not be read past: whether running 12.3 on an unconverted 11.6 datadir ever actually breaks.** It did not break here. MariaDB calls the upgrade required; we measured that it is skipped and did **not** measure a consequence. **What would settle it, cheaply:** run the E3 edge, then restart the stack several times and exercise the app, watching for the entrypoint's own complaint to become an error — one guest, no new mechanism. **The fix, if the consequence is real, is one line of template env**, but setting `MARIADB_AUTO_UPGRADE` fleet-wide is a change to how every MariaDB app upgrades and is the operator's call, not a quiet edit. Owner: **CC measures, VIKTOR rules on the fleet-wide env.** `audits/SPIKE-upgrade-test-2026-09-06.md` §4 | **READY — rank P2-MEDIUM; owner: CC measures, VIKTOR rules** | +| **R-460** | **[P3-LOW] BookStack's FILE half cannot be seeded or verified without a browser, so its upgrades can only ever be auto-proven for the DATABASE.** MEASURED 2026-09-06 while building the R-449 harness. BookStack's API needs a token that is only mintable through its web UI, and its HTTP login is unusable headlessly for a second, independent reason: `APP_URL` comes from the template as `https://${SUBDOMAIN}.${DOMAIN}`, so the app marks its session and XSRF cookies **`secure`**; curl over plain http stores neither and **every login POST returns 419 Page Expired**, which looks exactly like a wrong password. The container serves no TLS. **The database half IS provable** — the harness seeds with `php artisan bookstack:create-admin` and reads back with a DIFFERENT artisan command that must find the record, carrying its own negative control on every call. **What is unprovable is an uploaded image or attachment**, i.e. exactly the half a customer would notice. **THIS IS A FACT ABOUT THE APP, NOT A DEFECT IN THE HARNESS**, and it is recorded because Slice 6 needs to know which apps can be auto-verified and which can only be partly verified — nobody had that list before. **Deliberately NOT worked around:** planting a file in the volume would make the test pass while proving nothing, which is R-156's exact failure. **What would remove it:** a headless token route (upstream), or accepting a browser-driven step for this app alone, which DooPlex cannot run. Owner: **CC.** `audits/SPIKE-upgrade-test-2026-09-06.md` §6 | **READY — rank P3-LOW; owner: CC** | +| **R-461** | **[P3-LOW] `runbooks/target-selection.md` names a venue that does not exist and fences a fixture that is gone.** MEASURED 2026-09-06 on demo-hp while siting the R-449 guest. (a) The runbook says to put VM disks on a dir storage at **`/mnt/nvme-1tb`, at its root**. **There is no `/mnt/nvme-1tb`** — the 1 TB NVMe is mounted at **`/mnt/hdd_1`**, which is the enrolled user-data drive and the `felhom-backup` target, i.e. the same disk under a different path. The instruction's REASON is still exactly right (`local-lvm` is an over-subscribed thin pool backing the live guest 9201, and this run kept off it — `local-lvm` read **30.50 % before and after**), so the fence held; only its address is stale. (b) The runbook forbids destroying **`drill-r50` (VM 300)**, "the only drift fixture (R-93)". **`qm list` returns nothing on demo-hp** — there are no VMs at all. **The fence currently protects nothing, and R-93's premise that a drift fixture exists is false.** **Why it is a row and not a quiet edit:** a runbook that names a path nobody can find is one a session works around, and working around a safety instruction is how the instruction stops being followed. Both halves need checking against the box before the text is changed — (b) in particular may mean R-93 should be closed or reopened as "the drift fixture is gone", which is a different fact from "do not destroy it". Owner: **CC.** `audits/SPIKE-upgrade-test-2026-09-06.md` §7 | **READY — rank P3-LOW; owner: CC** | +| **R-462** | **[P2-MEDIUM] Widen the upgrade harness beyond three apps — and the cost is dominated by FIXTURES, not by machine time.** The R-449 harness works and is proven by a red negative control (`audits/SPIKE-upgrade-test-2026-09-06.md` §1). **Costed with this run's REAL numbers rather than an estimate:** a successful edge takes **6.4 s – 305.1 s, median 71.8 s**; a FAILING edge takes **556 s**, roughly **8×**, because a negative is only honest if it waits out the full settle window; 3 apps / 11 images cost **5.07 GB**, so 53 apps naively extrapolate to **~90 GB** and, at the median, about an hour of harness time for one edge each. **THAT EXTRAPOLATION UNDERSTATES THE REAL COST BY AN ORDER OF MAGNITUDE, and that is the point of this row.** Two of the three apps needed a bespoke non-browser seed route; one needed two attempts and a discarded approach; one (bookstack) can only ever be half-proven (R-460). **Fixture time scales with apps and does not amortise.** **The decision this row is really asking for is scope, not schedule:** all 53, or only the apps a customer would lose data from, or only apps whose catalog transition is a MAJOR. **Recommended shape, NOT a design — the operator picks:** start with the apps that carry a database, because §3 measured that the abort question only ever bites there. Owner: **VIKTOR rules on scope, CC implements.** `audits/SPIKE-upgrade-test-2026-09-06.md` §5 | **READY — rank P2-MEDIUM; owner: VIKTOR rules on scope, CC implements** |