diff --git a/CONTEXT.md b/CONTEXT.md index 604289ec..b27793a0 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -16,6 +16,13 @@ > and holds nothing of its own; this file does hold its own content, namely the standing rulings below. +> **2026-09-30 (late evening) — both rulings built.** Decision 52: catalog `6a3ead9` (re-test entries `from` == `to` + +> `digest_from` + `box_evidence`, gated with decoys; `upgrade-test.py --retest`; the monthly command +> `scripts/retest-floating.py`; runbook `runbooks/monthly-floating-retest.md`; proven end to end on 9202 through the leg). +> Decision 53 + R-741: controller v0.284.2 (image retention with a one-time sweep; the install hold), floor 0.284.2, +> golden 0.284.2 baked + vouched (gate OK). wger JWT pair `45d8482`. wanderer runs on the bench (override), no step yet. +> Rows R-743..R-747 opened; R-736/737/740/741 closed. Report: `REPORT-night-rulings-2026-09-30.md`. + > **Rulings 2026-09-30 (late evening, operator) — `09` §3 decisions 52 and 53, recorded before the work.** **52 (R-740 A):** > a box takes a same-tag security fix at night only after the catalog re-tested that tag at the new digest on both venues > and wrote it as a ladder step; database and redis lines first; one command, run monthly. **53 (R-736 A):** a box keeps, diff --git a/REPORT-night-rulings-2026-09-30.md b/REPORT-night-rulings-2026-09-30.md new file mode 100644 index 00000000..099d4d69 --- /dev/null +++ b/REPORT-night-rulings-2026-09-30.md @@ -0,0 +1,54 @@ +# REPORT — the operator's two rulings built (2026-09-30, late evening) + +Evidence: `documentation/audits/night-rulings-2026-09-30/` · golden: `documentation/tests/golden-0.284.2-2026-09-30/`. +Architecture read: `09` §3 decisions 13, 14, 17, 30, 40, 45–47, §5.3, §6.4 parts 4–7, §6.5; `07` (R-698); `01` §5. +Baselines (live Gitea 21:48): controller `d48da6c` (0.283.1), agent `d766666` (0.138.0), felhom.eu `42bf40b`, catalog +`d181165`. Register 377 rows by the method "every table row that starts with an R-id, bold or not, unique ids" (the +reviewer's regex and mine agree on 377 today). + +## The Part table + +| Part | done / not done / changed | why | +|---|---|---| +| **Rulings 52, 53** | **done** — recorded first (`09` §3, `07`, CONTEXT; decision 30 note) `1de0f7c` | before any work | +| **A — same-tag re-tests** | **done** — catalog `6a3ead9`: the entry shape, the gates (+8 decoys, seen red), `upgrade-test.py --retest`, the ONE command `retest-floating.py` | spike note `A/A0-spike.md` first: the move gate never looked at a re-test (it changes `.felhom.yml` only) | +| A4 end to end on 9202 | **done** — docmost at an older `redis:7-alpine` digest, "run tonight's chain now", the leg's `step pressed … step ended done after 95.0 s`, new digest running, read back, badge current | the first attempt (outline) stopped at outline's fixture on both venues (R-744) | +| A5 the real run | **done — nothing to re-test on the engine lines**; two EXACT tags were rebuilt upstream (R-743) | `--engines-only` is decision 52's start | +| A6 scheduling | **changed: a runbook, not a cron job** — `runbooks/monthly-floating-retest.md`; a standing monthly step in STATUS | it needs a fresh bench, 9202 on the drill, a drill force-reset, pushes to the live catalog | +| **B — image retention** | **done — controller v0.284.2** (0.284.0 and 0.284.1 never floored) | two faults found LIVE on 9202: the Remove button runs `RemoveStack` (only `DeleteStack` was wired); `docker image ls` without `-a` hides the untagged digest-pulled app images | +| B3 one-time sweep | **done** — 9202 26.6 → 5.7 GB (26 images), demo-hp 24.3 → 13.5 GB (24), the N100 6.15 → 6.07 GB (1); every app healthy | old controller images stay by design (R-745) | +| B4 red-proofs | **done at unit level** (shared image, undo's image, stopped app's compose, update in flight, unreadable keep set, both wirings, untagged images); **the live "update → failure → undo with no pull" was NOT run** | no failing edge was built tonight; the previous image is in the keep set (red-proofed) | +| **C — the install hold** | **done — controller v0.284.x** — the setup gate's door in front of an `after_install` app until the login is replaced | the brief expected 404 until the change; it is the gate's 401 (the household still passes) | +| C3 live on 9202 | **done** — calibre-web 0 of 192 stranger tries with the default login got in; mealie 0 of 97 | the positive control with the generated password was NOT obtained (a backup stopped calibre-web at that moment; mealie locked itself — R-747) | +| **D1 wger's key** | **done** — catalog `45d8482`; bench: right password 200 with a token that reads the API; wrong password **400** (not 401) | not proven on a box | +| **D2 wanderer** | **changed: the bench runs it; no step** — web/sign-up/login 200 with a bench-only override; meilisearch v1.54 needs `MEILI_UPGRADE_DB=true` (then the indexes survive) | a list could not be created (PocketBase refused) — no fixture, so no step (R-739) | +| **E — release, floor, golden** | **done** — floor **0.284.2** (MinAgent 0.131.0 declared), both demo boxes on 0.284.2 within a minute; golden **0.284.2** baked, round-trip identical, vouched (agent 0.138.0, min_agent 0.131.0); the gate prints **OK** (not WAIVED) | no agent release: MinAgent unchanged | + +## Claims in the brief that turned out wrong (or right) + +1. **"The box needs no change to press a same-tag re-test"** — **right** (unit walk; the e2e leg on 9202). +2. **"No tested digest differs from the registry today"** — right for the database/redis lines; **wrong for two exact + tags**: `nextcloud:34.0.4-apache`, linuxserver `sonarr:4.0.20` (R-743). +3. **"`--rmi local` never removes a registry-tagged image"** — right, and **the Remove button does not even use it**: + `RemoveStack` runs `down --volumes`; only the older `DeleteStack` had `--rmi local`. +4. **"Images are shared between apps"** — right: `postgres:18-alpine` and `redis:7-alpine` served docmost and paperless; + a remove of docmost kept both. +5. **"The route is published before `after_install` runs"** — right (measured earlier; now held). +6. **"Wanderer's web server needs the public DB name"** — right: `PUBLIC_POCKETBASE_URL` is the only address both images read. +7. Also: the brief expected 404 during the hold (it is the gate's 401) and 401 for wger's wrong password (it is 400); + "hub — read only" and Part E's floor + vouch conflict — the two form saves were done, nothing else on the hub. + +## Rows + +**377 → 383** (every `| **R-[letter]** |` row; the register-shape gate skipped the 3 lettered ids until tonight — R-748). Opened R-743 (exact tags rebuilt), R-744 (outline fixture at 1.10.1), R-745 (old controller images), +R-746 (`image_digest.resolve` ignores a digest), R-747 (mealie lockout by strangers), R-748 (the gate's lettered-id blind spot — fixed). Closed R-736, R-737, R-740, R-741, R-748. +Narrowed R-739, R-698, R-446. + +## Teardown + +- **Machine:** 9202 back on the live catalog (`repo_url` read back), its drill `update:` override removed, the same + containers as at the start, controller 0.284.2; the apps this run installed removed through the product. Bench LXC 9401 + destroyed with its template. Drill VM: CT 9100 destroyed, secrets shredded, qemu exited, `drill.qcow2` reverted to + `virgin`. The drill catalog reset to live (`6a3ead9`), image lines identical. +- **Host:** demo-hp `pct list` = 9201, 9202 (as at the start); the N100 untouched except the floor's controller update. +- **Hub:** two form saves only — the floor (0.284.2, MinAgent 0.131.0) and the vouch (golden 0.284.2). diff --git a/STATUS.md b/STATUS.md index bd7c7182..678ebd1d 100644 --- a/STATUS.md +++ b/STATUS.md @@ -2,65 +2,54 @@ **Ready for the first real tester (Tester-2): yes. You confirmed the tunnel route and the connect mails (2026-09-30).** -**Updated 2026-09-30 (evening). Both demo boxes run controller 0.283.1 and host agent 0.138.0. Hub 0.126.0. New installs get golden 0.283.1 with agent 0.138.0. No controller, agent or hub release today.** +**Updated 2026-09-30 (late evening). Both demo boxes run controller 0.284.2 and host agent 0.138.0. Hub 0.126.0. New installs get golden 0.284.2 with agent 0.138.0.** -**Tester-2 — read only, from the hub.** The customer record exists. Tester-2's box has not registered yet, so there are no apps to read. +**Tester-2 — read only, from the hub.** The customer record exists. Tester-2's box has not registered yet. -## One decision for you +## Your two decisions of tonight — built -**Question: should a box also take, at night, a security fix that the app's maker ships under the same name?** -Database images like `postgres:18-alpine` or `redis:7-alpine` get fixes without a new name. Seven of the eight such names -we use on Docker Hub got a new push in the last 30 days. +- **Security fixes under the same name (your A).** The catalog can now re-test a database image's new fix and record + it as a tested step. A box then takes it at night by itself. Tested from start to end on the scratch box: the night + run took the fix, the app kept its data, and the page went back to "up to date". **Today there was nothing to re-test** + for the database and redis images. +- **Old program files on the disk (your A).** A box now keeps each app's current program and the one before it, and + deletes the older ones. It never deletes one that another app still uses. On the first start it cleaned up once: + **the HP demo box freed 11 GB**, the scratch box 21 GB. All apps kept running. -Today **no box gets these fixes at all** — not at night, and not by the Update button either. The reason: the catalog -only records a test when the name changes, so the box never learns that a newer, tested image exists. The page says -"up to date". +## What else I did, and it worked -- **Option A — the night takes a tested fix.** The catalog re-tests the same name at the new image (bench and scratch - box, as today), and records it. The box then takes it at night by itself, with its backup and undo. Measured: the box - already does this with no change; only the catalog side is new work (about one evening to build). Cost after that: - about 20 app re-tests a month, machine time only, run by day. -- **Option B — keep it manual, say so honestly.** No new work. The fixes arrive only when we move an app to a new name. - Database engines rarely change name, so their fixes may wait for months. I would correct the text of the earlier - decision, which says the night would take them. - -**If nothing is decided:** nothing breaks. Database images stay at the build of the day we tested them. -**Who is blocked:** nobody today. **My recommendation: A**, database and redis names first — that is where security -fixes land, and the box side already works. - -## What I did, and it worked - -- **More apps update themselves at night: 35 now (plus 3 when a full copy exists), up from 30 (+2) at the start of the - evening.** New: gitea, wger, crafty-controller, uptime-kuma, zipline. calibre-web counts as "with a full copy", because its - update rewrites the book library's own database file. -- **Twelve updates tested and published**, each on the test bench and on the scratch box: calibre-web, gitea, wger, - crafty-controller, uptime-kuma, zipline (their first tested updates; zipline in two steps), and emby, ghost, home-assistant, outline, rallly. - Each of these apps is now on its maker's newest version in its line. -- **calibre-web (books) and gitea now have tests.** calibre-web: a book goes in through its Upload button and is read - back. gitea: its first-run form is filled in the way a household does it. -- **immich's older in-between update is fixed.** It still gave the database 512 MB and it does reload the big list of - places. Tested again with 768 MB and no swap: 0 kills. No box runs immich today. +- **The short login risk is closed.** After an install, an app with a known default password is now closed to + strangers until the box has changed that password. Tested on two apps: no stranger got in. +- **wger's phone app can log in** now (its missing key is made at the first start). +- **New controller 0.284.2** is on both demo boxes, and **a new golden 0.284.2** is baked and vouched. The golden check + is green (not just waived). ## What broke, and what I did -- **wger: an update would have broken it.** After the update the app looked fine, but nobody could log in. Its database - was never upgraded, because the catalog forgot one setting. **Fixed** in the catalog, and tested again: it works. No - box runs wger. -- **zipline: its newest version cannot be reached in one jump.** The scratch box saw it fail and **put the old version - back by itself in 20 seconds** — the undo worked on a real failure. Going through the version in between works; that - is now the path. -- **The scratch box's disk filled with old app images.** Removing an app or updating it never deletes the old image. - On a household box this would slowly fill the disk until updates are refused. Written down; not fixed today. -- **For a few seconds after a fresh install, calibre-web's well-known default login works.** The box changes it right - after, but the app is already reachable. Written down; not fixed today. -- **wanderer cannot run on the test bench**, so it was not tested. Written down. +- **The clean-up first missed most images.** Two mistakes, both found on the scratch box before any demo box got the + release: it did not run on the Remove button, and it could not see images stored without a name. Both fixed and + tested. That is why the version is 0.284.2 (0.284.0 and 0.284.1 never went out). +- **wanderer** can now run on the test bench, but its update is not tested yet: I could not create test data in it. + Also found: its search engine needs one extra setting before any update, or it refuses to start. +- **Two exact versions were rebuilt by their makers under the same name** (nextcloud and sonarr). The monthly re-test + finds them. Tonight I only did the database images, as you ruled. +- **mealie locks the account after five wrong passwords** — also when a stranger types them. Written down. +- **Old controller versions** still pile up on each box (about 50). Your rule covered apps only. Written down. -**Rows.** Tonight: 3 closed (all three found and fixed tonight), 6 narrowed, 8 opened. The list went from 369 to 377 rows. *(The brief said 366; the -afternoon session had already added three.)* +**Rows.** Tonight: 5 closed, 3 narrowed, 6 opened. The list went from 377 to 383 rows (counted with every row, including three whose number has a letter — the old check skipped those). ## What needs you -1. **The decision above** (same-name security fixes). If you do nothing, nothing changes. -2. **The image clean-up on a box** will need your word on what the box keeps (the running image and the one before it, - for the undo). If you do nothing, disks fill slowly; nothing breaks tonight. -3. **No golden bake is due.** The weekly bake stays around 7 October. +1. **Who presses the monthly re-test?** It is one command, but it needs the test bench and the scratch box set up + first, so it cannot run by itself (the steps are written down). Options: CC does it in a monthly session, or you + start it. If nobody presses it, same-name security fixes wait until the next time we move that app. +2. **Should the monthly re-test cover every app, or only databases?** Exact versions get rebuilt too (nextcloud, + sonarr). If you do nothing, only databases are re-tested. +3. **Old controller versions:** keep only the current and the previous one? If you do nothing, about 150 MB per release + keeps piling up. +4. **No golden bake is due** — tonight's bake carries the newest controller. + +## Standing steps + +- **Monthly:** the same-name security re-test (runbook `monthly-floating-retest.md`). +- **Weekly:** the golden bake (around 7 October). diff --git a/documentation/architecture/09-update-architecture.md b/documentation/architecture/09-update-architecture.md index e9e4246e..7ef841fa 100644 --- a/documentation/architecture/09-update-architecture.md +++ b/documentation/architecture/09-update-architecture.md @@ -587,6 +587,12 @@ R-636's louder repeated alarm. reached no box at all (R-740: the catalog never recorded a same-tag re-test, and the leg skips a digest-only change); measured that the box's leg presses such an entry with no controller change. Decision 30's cost line ("or the automatic leg") is corrected by the dated note there; decision 30's ruling is unchanged. + **Outcome (2026-09-30 late):** catalog `6a3ead9` — the re-test entry (`from` == `to`, `digest_from`, `box_evidence`), + its gates and decoys, `upgrade-test.py --retest`, and the ONE command `scripts/retest-floating.py`. Proven end to end + on 9202: docmost at an older `redis:7-alpine` digest → the re-test → "run tonight's chain now" → the leg pressed the + step, the new digest ran, the data read back, the badge went back to current. The box needed no change. No + database/redis line differs today; two exact tags do (R-743). The monthly run is a runbook, not a cron job + (`runbooks/monthly-floating-retest.md`). 53. **A box keeps, per app service, the image it runs now and the image before it (the undo's); it deletes older images of that app by itself** — *operator ruling 2026-09-30 evening (R-736, option A).* It never deletes an image that any container (running or stopped) or any installed app's compose still names. Removing an app deletes that app's @@ -595,6 +601,9 @@ R-636's louder repeated alarm. that filled the Docker disk until the box refused an install (R-736). **Cost, stated:** a restore to a version older than the previous one re-pulls it — as every restore already does (R-698: a backup stores the image's name, not the image). + **Outcome (2026-09-30 late):** controller v0.284.2 (floor 0.284.2; 0.284.0 and 0.284.1 never floored — two wiring + faults found live on 9202). The one-time sweep: 9202 26.6 → 5.7 GB, demo-hp 24.3 → 13.5 GB, the N100 6.15 → 6.07 GB, + every app healthy. Old CONTROLLER images are outside this decision and stay (R-745). ### 2026-09-30 (day) — operator notes, recorded before the work diff --git a/documentation/audits/night-rulings-2026-09-30/A/A0-spike.md b/documentation/audits/night-rulings-2026-09-30/A/A0-spike.md new file mode 100644 index 00000000..32a68062 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/A0-spike.md @@ -0,0 +1,18 @@ +# Part A spike — can a same-tag re-test be written and gated? (2026-09-30 late evening, read from source, catalog d181165) + +**The entry shape:** `from` and `to` name the SAME refs; `digest` = the digest the re-test proved (the registry's +current one); NEW field `digest_from` = the digest it was tested FROM (the previous entry's digest for those services); +`evidence` (bench) AND `box_evidence` (9202) both present. Everything else as any proven entry (memory watch, marks). + +| piece | today | must accept | must refuse | +|---|---|---|---| +| `upgrade-test.py` writer | `--move` exits "nothing moves" (L919); `--write-ladder` compares the template with the verdict's `from` literally | a verdict whose `from`/`to` carry `ref@old` / `ref@new` → written as plain refs + `digest_from` | a verdict whose new digest equals the old; a digest the registry no longer serves (resolved at write time); a missing box verdict | +| `ladder.check_entry` | a `from == to` entry passes with no further check | `from == to` with `digest_from` for every service and ≥1 service whose digest differs; `box_evidence` present | no `digest_from`; no digest differs ("no new digest"); no `box_evidence` ("without both venues") | +| `check-test-record.py` (static) | rules 1–4 | rule 2 continuity holds (`from` = previous `to`); rule 4 — the superseded head's step file has the same key as the re-test's `to`, and the SAME images: consistent | NEW rule 2b: a re-test's `digest_from` must equal the previous entry's `digest` (a re-test from somewhere else) | +| `check-test-record-move.py` (history + registry) | looks ONLY at templates whose `docker-compose.yml` changed — a re-test changes `.felhom.yml` only, so it would pass UNCHECKED | a new re-test entry whose digest the registry serves now | a new re-test entry whose digest the registry no longer serves | + +**The box (unit walk, `more-night-apps-2026-09-30/C/C2`):** the leg presses a `from == to` entry with no controller change, +renders the new digest, the next night reads current. One consequence, stated: a box one step BEHIND the old head pins the +old head's step file and renders the digest of the NEWEST entry whose `to` is those refs — the re-test's — so it lands on +the new digest directly (the re-test proved old → new for that tag; the combination older-step → new digest is not +separately tested). Accepted: the tag is the same line, and the re-test is the proof of that line at that digest. diff --git a/documentation/audits/night-rulings-2026-09-30/A/A1-decoys-red.txt b/documentation/audits/night-rulings-2026-09-30/A/A1-decoys-red.txt new file mode 100644 index 00000000..5954e967 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/A1-decoys-red.txt @@ -0,0 +1,15 @@ +# decoy red-proof: the three new rules switched off (ladder.check_entry re-test block, check-test-record rule 2b, the move gate's re-test pass) +-- test-record: a same-tag re-test (decision 52) + XX GENUINE: head + a re-test from its digest, both venues rc=1 (expected 0) + XX FACT: a re-test with NO new digest rc=1 (expected 1) + XX FACT: a re-test without the box venue rc=1 (expected 1) + XX FACT: a re-test FROM a digest the previous entry never tested rc=1 (expected 1) + XX DECOY: a re-test that names digest_from only in its evidence text rc=0 (expected 1) + ok GENUINE: a re-test whose digest the registry serves now rc=0 (expected 0) + XX FACT: a re-test whose digest the registry no longer serves rc=0 (expected 1) + XX FACT: a re-test with no new digest reaches the move gate too rc=0 (expected 1) + ok probe-measured: genuine komga note passes rc=0 (expected 0) +test-record-move gate — range HEAD~1..HEAD: 0 compose file(s) changed + +# restored: +restored rc=0 diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/E0-repoint-drill.txt b/documentation/audits/night-rulings-2026-09-30/A/e2e/E0-repoint-drill.txt new file mode 100644 index 00000000..f2c89786 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/E0-repoint-drill.txt @@ -0,0 +1,10 @@ +git: + branch: main + repo_url: https://gitea.dooplex.hu/admin/app-catalog-drill.git + sync_interval: 15m + token: + username: "admin" +hub: +update: + health_timeout: 90s + diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/E1-drill-old-digest.txt b/documentation/audits/night-rulings-2026-09-30/A/e2e/E1-drill-old-digest.txt new file mode 100644 index 00000000..5159a7c6 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/E1-drill-old-digest.txt @@ -0,0 +1 @@ +995a8e1 DRILL e2e (decision 52): outline's head says redis:7-alpine was tested at an OLDER digest c35af3bb (the month-ago state) diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/E1b-drill-old-digest-docmost.txt b/documentation/audits/night-rulings-2026-09-30/A/e2e/E1b-drill-old-digest-docmost.txt new file mode 100644 index 00000000..14139185 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/E1b-drill-old-digest-docmost.txt @@ -0,0 +1 @@ +1189d86 DRILL e2e (decision 52): docmost's head says redis:7-alpine was tested at an OLDER digest c35af3bb diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/E2-writer.txt b/documentation/audits/night-rulings-2026-09-30/A/e2e/E2-writer.txt new file mode 100644 index 00000000..4ed9bd58 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/E2-writer.txt @@ -0,0 +1,17 @@ +STEP docmost: the superseded step {'docmost': 'docmost/docmost:0.96.0', 'docmost-postgres': 'postgres:18-alpine', 'docmost-redis': 'redis:7-alpine'} keeps its definition at steps/01ae17194c827ab5.yml +STEP docmost: … and its .felhom.yml at steps/01ae17194c827ab5.felhom.yml +WROTE docmost: RE-TEST {'docmost': 'docmost/docmost:0.96.0', 'docmost-postgres': 'postgres:18-alpine', 'docmost-redis': 'redis:7-alpine'} -> {'docmost': 'docmost/docmost:0.96.0', 'docmost-postgres': 'postgres:18-alpine', 'docmost-redis': 'redis:7-alpine'} peak 80.5% marks {'files_may_change': False, 'needs_person': None, 'memory_tight': True} digest {'docmost': 'sha256:b56947fcfd08aab8fae12a377e1792784786adbf8b96e4281f14ef4fc072685a', 'docmost-postgres': 'sha256:77f585114c32fbca283dc835b0596f4e52b51b4c6662d7810b2f4084f60a1873', 'docmost-redis': 'sha256:c35af3bbcef51a62c8bae5a9a563c6f1b60d7ebaea4cb5a3ccbcc157580ae098'} -> {'docmost': 'sha256:b56947fcfd08aab8fae12a377e1792784786adbf8b96e4281f14ef4fc072685a', 'docmost-postgres': 'sha256:77f585114c32fbca283dc835b0596f4e52b51b4c6662d7810b2f4084f60a1873', 'docmost-redis': 'sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499'} +test-record gate — 1 template(s) read, 1 carry a ladder, 0 convicted +test-record-move gate — range HEAD~1..HEAD: 0 compose file(s) changed +## the real registry: +test-record-move gate — range HEAD~1..HEAD: 0 compose file(s) changed +rc=0 +## positive control (a table = today's registry): + registry answers come from /tmp/good.json, NOT the registry (decoy tests only) +test-record-move gate — range HEAD~1..HEAD: 0 compose file(s) changed +rc=0 +## negative control (the registry moved again): + registry answers come from /tmp/moved.json, NOT the registry (decoy tests only) +test-record-move gate — range HEAD~1..HEAD: 0 compose file(s) changed +REFUSED docmost: re-test docmost-redis redis:7-alpine: the registry serves sha256:dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd, the re-test says sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499 — the re-tested image is not the image a box would pull +rc=1 diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/bench-outline-attempt-fixture-fault/RT-outline.log b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench-outline-attempt-fixture-fault/RT-outline.log new file mode 100644 index 00000000..f3f5a012 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench-outline-attempt-fixture-fault/RT-outline.log @@ -0,0 +1,37 @@ +[20:20:51] scratch drive folders cleared before FROM (R-656): none existed +[20:20:51] RT-outline: deploying outline at FROM {'outline': 'outlinewiki/outline:1.10.1', 'outline-postgres': 'postgres:18-alpine', 'outline-redis': 'redis:7-alpine@sha256:c35af3bbcef51a62c8bae5a9a563c6f1b60d7ebaea4cb5a3ccbcc157580ae098'} +[20:22:15] FROM settled=True in 31.2s :: {"outline": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "outline-postgres": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "outline-redis": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}} +[20:22:15] fixture: the BOX walk's own (Outline), through upgrade_boxport +[20:22:16] outline: installation.create http=302 +[20:22:16] INCONCLUSIVE — no non-browser seed route. Nothing was planted by hand. +{ + "harness_version": 4, + "edge": "RT-outline", + "app": "outline", + "note": "re-test of the same tag at a new digest (decision 52): outline-redis sha256:c35af3bbcef5 -> sha256:858f009f9709", + "from": { + "outline": "outlinewiki/outline:1.10.1", + "outline-postgres": "postgres:18-alpine", + "outline-redis": "redis:7-alpine@sha256:c35af3bbcef51a62c8bae5a9a563c6f1b60d7ebaea4cb5a3ccbcc157580ae098" + }, + "to": { + "outline": "outlinewiki/outline:1.10.1", + "outline-postgres": "postgres:18-alpine", + "outline-redis": "redis:7-alpine@sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499" + }, + "verdict": "inconclusive", + "seed_read_before": false, + "seed_read_after": false, + "healthy_after": false, + "migration_observed": null, + "abort": "not-attempted", + "abort_detail": "no non-browser seed route \u2014 tried: no csrfToken cookie from GET /home", + "engine_state_after": null, + "memory": null, + "marks": [], + "duration_s": 84.4, + "measured_at": "2026-09-30T20:22:16Z", + "evidence": "evidence/RT-outline", + "scratch_cleared": [], + "total_s": 84.4 +} diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/bench-outline-attempt-fixture-fault/evidence/RT-outline/compose-final.log b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench-outline-attempt-fixture-fault/evidence/RT-outline/compose-final.log new file mode 100644 index 00000000..edfb80af --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench-outline-attempt-fixture-fault/evidence/RT-outline/compose-final.log @@ -0,0 +1,715 @@ +outline-redis | 1:C 30 Sep 2026 22:21:33.972 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. +outline-redis | 1:C 30 Sep 2026 22:21:33.972 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo +outline-redis | 1:C 30 Sep 2026 22:21:33.972 * Redis version=7.4.0, bits=64, commit=00000000, modified=0, pid=1, just started +outline-redis | 1:C 30 Sep 2026 22:21:33.972 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf +outline-redis | 1:M 30 Sep 2026 22:21:33.972 * monotonic clock: POSIX clock_gettime +outline-redis | 1:M 30 Sep 2026 22:21:33.973 * Running mode=standalone, port=6379. +outline-redis | 1:M 30 Sep 2026 22:21:33.974 * Server initialized +outline-redis | 1:M 30 Sep 2026 22:21:33.974 * Ready to accept connections tcp +outline | {"label":"lifecycle","level":"info","message":"Note: Restricting process count to 1 due to use of collaborative service without REDIS_COLLABORATION_URL"} +outline | {"label":"lifecycle","level":"info","message":"Memory constraint of 768MB detected, defaulting each service process to a 614MB heap limit"} +outline | ⟐ injected env (0) · dotenvx@1.75.1 +outline | (node:15) ExperimentalWarning: localStorage is not available because --localstorage-file was not provided. +outline | (Use `node --trace-warnings ...` to show where the warning was created) +outline | {"label":"database","level":"info","message":"Running migrations…"} +outline | {"label":"database","level":"info","message":"Migrating 20160619080644-initial.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20160619080644-initial.js in 0.055s"} +outline | {"label":"database","level":"info","message":"Migrating 20160622043741-add-parent-document.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20160622043741-add-parent-document.js in 0.013s"} +outline | {"label":"database","level":"info","message":"Migrating 20160626063409-add-indexes.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20160626063409-add-indexes.js in 0.041s"} +outline | {"label":"database","level":"info","message":"Migrating 20160626175224-add-revisions.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20160626175224-add-revisions.js in 0.028s"} +outline | {"label":"database","level":"info","message":"Migrating 20160711071958-search-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20160711071958-search-index.js in 0.021s"} +outline | {"label":"database","level":"info","message":"Migrating 20160726061511-atlas-creator.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20160726061511-atlas-creator.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20160812145029-document-atlas-soft-delete.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20160812145029-document-atlas-soft-delete.js in 0.015s"} +outline | {"label":"database","level":"info","message":"Migrating 20160814083127-paranoia-indeces.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20160814083127-paranoia-indeces.js in 0.059s"} +outline | {"label":"database","level":"info","message":"Migrating 20160814095336-add-document-createdById.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20160814095336-add-document-createdById.js in 0.013s"} +outline | {"label":"database","level":"info","message":"Migrating 20160814111419-add-document-collaboratorIds.js…"} +outline-postgres | The files belonging to this database system will be owned by user "postgres". +outline-postgres | This user must also own the server process. +outline-postgres | +outline-postgres | The database cluster will be initialized with locale "en_US.utf8". +outline-postgres | The default database encoding has accordingly been set to "UTF8". +outline-postgres | The default text search configuration will be set to "english". +outline-postgres | +outline-postgres | Data page checksums are enabled. +outline-postgres | +outline-postgres | fixing permissions on existing directory /var/lib/postgresql/18/docker ... ok +outline-postgres | creating subdirectories ... ok +outline-postgres | selecting dynamic shared memory implementation ... posix +outline | {"label":"database","level":"info","message":"Migrated 20160814111419-add-document-collaboratorIds.js in 0.013s"} +outline | {"label":"database","level":"info","message":"Migrating 20160815142720-app-collection-urlId.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20160815142720-app-collection-urlId.js in 0.014s"} +outline | {"label":"database","level":"info","message":"Migrating 20160816082738-add-revision-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20160816082738-add-revision-index.js in 0.014s"} +outline | {"label":"database","level":"info","message":"Migrating 20160824061730-add-apikeys.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20160824061730-add-apikeys.js in 0.015s"} +outline | {"label":"database","level":"info","message":"Migrating 20160824062457-add-apikey-indeces.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20160824062457-add-apikey-indeces.js in 0.017s"} +outline | {"label":"database","level":"info","message":"Migrating 20160911230444-user-optional-slack-id.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20160911230444-user-optional-slack-id.js in 0.034s"} +outline | {"label":"database","level":"info","message":"Migrating 20160911232911-user-unique-fields.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20160911232911-user-unique-fields.js in 0.042s"} +outline | {"label":"database","level":"info","message":"Migrating 20160911234928-user-password.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20160911234928-user-password.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20170603185012-add-collection-documentStructure-migration.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20170603185012-add-collection-documentStructure-migration.js in 0.017s"} +outline | {"label":"database","level":"info","message":"Migrating 20170604052346-add-views.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20170604052346-add-views.js in 0.018s"} +outline | {"label":"database","level":"info","message":"Migrating 20170604052347-add-stars.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20170604052347-add-stars.js in 0.018s"} +outline-postgres | selecting default "max_connections" ... 100 +outline-postgres | selecting default "shared_buffers" ... 128MB +outline-postgres | selecting default time zone ... Europe/Budapest +outline-postgres | creating configuration files ... ok +outline-postgres | running bootstrap script ... ok +outline-postgres | sh: locale: not found +outline-postgres | 2026-09-30 22:21:34.734 CEST [40] WARNING: no usable system locales were found +outline-postgres | performing post-bootstrap initialization ... ok +outline-postgres | syncing data to disk ... ok +outline-postgres | +outline-postgres | +outline-postgres | Success. You can now start the database server using: +outline | {"label":"database","level":"info","message":"Migrating 20170712055148-non-unique-email.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20170712055148-non-unique-email.js in 0.015s"} +outline | {"label":"database","level":"info","message":"Migrating 20170712072234-uniq-slack-id.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20170712072234-uniq-slack-id.js in 0.024s"} +outline | {"label":"database","level":"info","message":"Migrating 20170729215619-emoji.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20170729215619-emoji.js in 0.013s"} +outline | {"label":"database","level":"info","message":"Migrating 20170827182423-improve-references.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20170827182423-improve-references.js in 0.034s"} +outline | {"label":"database","level":"info","message":"Migrating 20170904202454-allow-null-username.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20170904202454-allow-null-username.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20171010042938-add-event.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20171010042938-add-event.js in 0.019s"} +outline | {"label":"database","level":"info","message":"Migrating 20171016012353-remove-collection-navigationtree.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20171016012353-remove-collection-navigationtree.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20171017055026-remove-document-html.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20171017055026-remove-document-html.js in 0.021s"} +outline | {"label":"database","level":"info","message":"Migrating 20171019071915-user-avatar-url.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20171019071915-user-avatar-url.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20171023064220-collection-color.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20171023064220-collection-color.js in 0.011s"} +outline-postgres | +outline-postgres | pg_ctl -D /var/lib/postgresql/18/docker -l logfile start +outline-postgres | +outline-postgres | initdb: warning: enabling "trust" authentication for local connections +outline-postgres | initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. +outline-postgres | waiting for server to start....2026-09-30 22:21:35.789 CEST [46] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-musl, compiled by gcc (Alpine 15.2.0) 15.2.0, 64-bit +outline-postgres | 2026-09-30 22:21:35.793 CEST [46] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" +outline-postgres | 2026-09-30 22:21:35.810 CEST [52] LOG: database system was shut down at 2026-09-30 22:21:35 CEST +outline-postgres | 2026-09-30 22:21:35.819 CEST [46] LOG: database system is ready to accept connections +outline-postgres | done +outline-postgres | server started +outline-postgres | CREATE DATABASE +outline-postgres | +outline-postgres | +outline-postgres | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/* +outline-postgres | +outline-postgres | waiting for server to shut down....2026-09-30 22:21:36.018 CEST [46] LOG: received fast shutdown request +outline-postgres | 2026-09-30 22:21:36.025 CEST [46] LOG: aborting any active transactions +outline-postgres | 2026-09-30 22:21:36.028 CEST [46] LOG: background worker "logical replication launcher" (PID 55) exited with exit code 1 +outline-postgres | 2026-09-30 22:21:36.029 CEST [50] LOG: shutting down +outline-postgres | 2026-09-30 22:21:36.035 CEST [50] LOG: checkpoint starting: shutdown immediate +outline-postgres | 2026-09-30 22:21:36.143 CEST [50] LOG: checkpoint complete: wrote 943 buffers (5.8%), wrote 3 SLRU buffers; 0 WAL file(s) added, 0 removed, 0 recycled; write=0.033 s, sync=0.052 s, total=0.114 s; sync files=303, longest=0.017 s, average=0.001 s; distance=4362 kB, estimate=4362 kB; lsn=0/1BA8858, redo lsn=0/1BA8858 +outline-postgres | 2026-09-30 22:21:36.178 CEST [46] LOG: database system is shut down +outline-postgres | done +outline-postgres | server stopped +outline-postgres | +outline | {"label":"database","level":"info","message":"Migrating 20171218043717-add-authentications.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20171218043717-add-authentications.js in 0.019s"} +outline | {"label":"database","level":"info","message":"Migrating 20171225143838-set-admins.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20171225143838-set-admins.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20180115021837-add-drafts.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20180115021837-add-drafts.js in 0.02s"} +outline | {"label":"database","level":"info","message":"Migrating 20180212033504-add-integrations.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20180212033504-add-integrations.js in 0.022s"} +outline | {"label":"database","level":"info","message":"Migrating 20180225203847-document-pinning.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20180225203847-document-pinning.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20180303193036-suspended-users.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20180303193036-suspended-users.js in 0.017s"} +outline | {"label":"database","level":"info","message":"Migrating 20180324214403-serializer-upgrade.js…"} +outline-postgres | PostgreSQL init process complete; ready for start up. +outline-postgres | +outline-postgres | 2026-09-30 22:21:36.260 CEST [1] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-musl, compiled by gcc (Alpine 15.2.0) 15.2.0, 64-bit +outline-postgres | 2026-09-30 22:21:36.260 CEST [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 +outline-postgres | 2026-09-30 22:21:36.261 CEST [1] LOG: listening on IPv6 address "::", port 5432 +outline-postgres | 2026-09-30 22:21:36.273 CEST [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" +outline-postgres | 2026-09-30 22:21:36.284 CEST [68] LOG: database system was shut down at 2026-09-30 22:21:36 CEST +outline-postgres | 2026-09-30 22:21:36.296 CEST [1] LOG: database system is ready to accept connections +outline | {"label":"database","level":"info","message":"Migrated 20180324214403-serializer-upgrade.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20180513041057-add-share-links.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20180513041057-add-share-links.js in 0.019s"} +outline | {"label":"database","level":"info","message":"Migrating 20180528233909-google-auth.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20180528233909-google-auth.js in 0.079s"} +outline | {"label":"database","level":"info","message":"Migrating 20180528233910-rename-serviceid.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20180528233910-rename-serviceid.js in 0.056s"} +outline | {"label":"database","level":"info","message":"Migrating 20180604182823-user-tracking.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20180604182823-user-tracking.js in 0.021s"} +outline | {"label":"database","level":"info","message":"Migrating 20180604191743-revoke-share-links.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20180604191743-revoke-share-links.js in 0.015s"} +outline | {"label":"database","level":"info","message":"Migrating 20180707220121-more-soft-delete.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20180707220121-more-soft-delete.js in 0.016s"} +outline | {"label":"database","level":"info","message":"Migrating 20180707231201-remove-passwords.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20180707231201-remove-passwords.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20180708231200-serviceid-null.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20180708231200-serviceid-null.js in 0.018s"} +outline | {"label":"database","level":"info","message":"Migrating 20180808061353-cleanup.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20180808061353-cleanup.js in 0.049s"} +outline | {"label":"database","level":"info","message":"Migrating 20180819054252-disable-sharing.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20180819054252-disable-sharing.js in 0.013s"} +outline | {"label":"database","level":"info","message":"Migrating 20181031015046-add-subdomain-to-team.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20181031015046-add-subdomain-to-team.js in 0.018s"} +outline | {"label":"database","level":"info","message":"Migrating 20181124000438-add-notifications.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20181124000438-add-notifications.js in 0.036s"} +outline | {"label":"database","level":"info","message":"Migrating 20181215192422-document-embeds.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20181215192422-document-embeds.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20181227001547-collection-permissions.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20181227001547-collection-permissions.js in 0.025s"} +outline | {"label":"database","level":"info","message":"Migrating 20190404035736-add-archive.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20190404035736-add-archive.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20190423051708-add-search-indexes.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20190423051708-add-search-indexes.js in 0.023s"} +outline | {"label":"database","level":"info","message":"Migrating 20190606035733-events.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20190606035733-events.js in 0.035s"} +outline | {"label":"database","level":"info","message":"Migrating 20190704070630-welcome-docs.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20190704070630-welcome-docs.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20190706213213-backlinks.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20190706213213-backlinks.js in 0.024s"} +outline | {"label":"database","level":"info","message":"Migrating 20190811231511-maintainers.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20190811231511-maintainers.js in 0.021s"} +outline | {"label":"database","level":"info","message":"Migrating 20191118023010-cascade-delete.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20191118023010-cascade-delete.js in 0.019s"} +outline | {"label":"database","level":"info","message":"Migrating 20191119023010-cascade-backlinks.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20191119023010-cascade-backlinks.js in 0.02s"} +outline | {"label":"database","level":"info","message":"Migrating 20191119023011-cascade-parent-documents.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20191119023011-cascade-parent-documents.js in 0.017s"} +outline | {"label":"database","level":"info","message":"Migrating 20191119023012-cascade-shares.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20191119023012-cascade-shares.js in 0.018s"} +outline | {"label":"database","level":"info","message":"Migrating 20191119023013-cascade-backlinks2.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20191119023013-cascade-backlinks2.js in 0.018s"} +outline | {"label":"database","level":"info","message":"Migrating 20191121035144-guest-invite.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20191121035144-guest-invite.js in 0.021s"} +outline | {"label":"database","level":"info","message":"Migrating 20191211044318-create-groups.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20191211044318-create-groups.js in 0.027s"} +outline | {"label":"database","level":"info","message":"Migrating 20191211044319-create-group-users.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20191211044319-create-group-users.js in 0.03s"} +outline | {"label":"database","level":"info","message":"Migrating 20191228031525-edit-presence.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20191228031525-edit-presence.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20200104233831-attachments.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20200104233831-attachments.js in 0.025s"} +outline | {"label":"database","level":"info","message":"Migrating 20200122083721-create-collection-groups.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20200122083721-create-collection-groups.js in 0.03s"} +outline | {"label":"database","level":"info","message":"Migrating 20200316040755-document-editor-version.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20200316040755-document-editor-version.js in 0.014s"} +outline | {"label":"database","level":"info","message":"Migrating 20200328175012-cascade-delete.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20200328175012-cascade-delete.js in 0.019s"} +outline | {"label":"database","level":"info","message":"Migrating 20200330053639-document-version.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20200330053639-document-version.js in 0.016s"} +outline | {"label":"database","level":"info","message":"Migrating 20200519032353-text-backup.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20200519032353-text-backup.js in 0.014s"} +outline | {"label":"database","level":"info","message":"Migrating 20200522054958-collection-icon.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20200522054958-collection-icon.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20200723055414-add-published-to-shares.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20200723055414-add-published-to-shares.js in 0.014s"} +outline | {"label":"database","level":"info","message":"Migrating 20200727051157-add-templates.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20200727051157-add-templates.js in 0.015s"} +outline | {"label":"database","level":"info","message":"Migrating 20200812170227-remove-collection-type.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20200812170227-remove-collection-type.js in 0.014s"} +outline | {"label":"database","level":"info","message":"Migrating 20200915010511-create-search-queries.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20200915010511-create-search-queries.js in 0.033s"} +outline | {"label":"database","level":"info","message":"Migrating 20200926204620-add-missing-indexes.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20200926204620-add-missing-indexes.js in 0.023s"} +outline | {"label":"database","level":"info","message":"Migrating 20201028043021-reverse-document-id-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20201028043021-reverse-document-id-index.js in 0.013s"} +outline | {"label":"database","level":"info","message":"Migrating 20201103050534-custom-domains.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20201103050534-custom-domains.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20201106122752-i18n.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20201106122752-i18n.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20201206210619-update-attachment-cols.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20201206210619-update-attachment-cols.js in 0.015s"} +outline | {"label":"database","level":"info","message":"Migrating 20201211080408-attachment-no-cascade.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20201211080408-attachment-no-cascade.js in 0.013s"} +outline | {"label":"database","level":"info","message":"Migrating 20201230031607-collection-sort.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20201230031607-collection-sort.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20210110143902-collection-rename-creator-id.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210110143902-collection-rename-creator-id.js in 0.032s"} +outline | {"label":"database","level":"info","message":"Migrating 20210208062816-disable-collection-sharing.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210208062816-disable-collection-sharing.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20210218111237-add-collection-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210218111237-add-collection-index.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20210226232041-authentication-providers.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210226232041-authentication-providers.js in 0.066s"} +outline | {"label":"database","level":"info","message":"Migrating 20210310051804-passport.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210310051804-passport.js in 0.034s"} +outline | {"label":"database","level":"info","message":"Migrating 20210314173941-isViewer.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210314173941-isViewer.js in 0.021s"} +outline | {"label":"database","level":"info","message":"Migrating 20210327005406-read-only-collections.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210327005406-read-only-collections.js in 0.056s"} +outline | {"label":"database","level":"info","message":"Migrating 20210418053152-share-last-viewed.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210418053152-share-last-viewed.js in 0.021s"} +outline | {"label":"database","level":"info","message":"Migrating 20210426055334-nested-document-sharing.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210426055334-nested-document-sharing.js in 0.02s"} +outline | {"label":"database","level":"info","message":"Migrating 20210430024222-marketing-tracking.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210430024222-marketing-tracking.js in 0.022s"} +outline | {"label":"database","level":"info","message":"Migrating 20210716064654-introduce-previousTitles.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210716064654-introduce-previousTitles.js in 0.017s"} +outline | {"label":"database","level":"info","message":"Migrating 20210716071454-search-index-previousTitles.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210716071454-search-index-previousTitles.js in 0.025s"} +outline | {"label":"database","level":"info","message":"Migrating 20210716162923-events-indexes.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210716162923-events-indexes.js in 0.031s"} +outline | {"label":"database","level":"info","message":"Migrating 20210730042450-remove-unused-indexes.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210730042450-remove-unused-indexes.js in 0.118s"} +outline | {"label":"database","level":"info","message":"Migrating 20210730044247-remove-backup-column.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210730044247-remove-backup-column.js in 0.049s"} +outline | {"label":"database","level":"info","message":"Migrating 20210730044248-create-realtime.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210730044248-create-realtime.js in 0.042s"} +outline | {"label":"database","level":"info","message":"Migrating 20210730210120-add-fileOperations.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210730210120-add-fileOperations.js in 0.234s"} +outline | {"label":"database","level":"info","message":"Migrating 20210915051740-collaborative-collections.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210915051740-collaborative-collections.js in 0.09s"} +outline | {"label":"database","level":"info","message":"Migrating 20210921031555-missing-cascades.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210921031555-missing-cascades.js in 0.056s"} +outline | {"label":"database","level":"info","message":"Migrating 20210923031555-missing-cascades.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20210923031555-missing-cascades.js in 0.032s"} +outline | {"label":"database","level":"info","message":"Migrating 20211003021903-missing-cascades.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20211003021903-missing-cascades.js in 0.019s"} +outline | {"label":"database","level":"info","message":"Migrating 20211015170955-add-defaultUserRole.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20211015170955-add-defaultUserRole.js in 0.015s"} +outline | {"label":"database","level":"info","message":"Migrating 20211107021900-missing-cascades.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20211107021900-missing-cascades.js in 0.052s"} +outline | {"label":"database","level":"info","message":"Migrating 20211217054419-integration-events.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20211217054419-integration-events.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20211218185045-remove-unused-indexes.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20211218185045-remove-unused-indexes.js in 0.019s"} +outline | {"label":"database","level":"info","message":"Migrating 20211218193004-documents-full-width.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20211218193004-documents-full-width.js in 0.014s"} +outline | {"label":"database","level":"info","message":"Migrating 20211221031430-create-pins.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20211221031430-create-pins.js in 0.027s"} +outline | {"label":"database","level":"info","message":"Migrating 20220117012250-add-starred-sorting.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220117012250-add-starred-sorting.js in 0.013s"} +outline | {"label":"database","level":"info","message":"Migrating 20220127000000-index-fixes.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220127000000-index-fixes.js in 0.051s"} +outline | {"label":"database","level":"info","message":"Migrating 20220129092607-add-defaultCollectionId.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220129092607-add-defaultCollectionId.js in 0.013s"} +outline | {"label":"database","level":"info","message":"Migrating 20220206225006-add-error-to-file-operation.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220206225006-add-error-to-file-operation.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20220305195830-create-comments.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220305195830-create-comments.js in 0.029s"} +outline | {"label":"database","level":"info","message":"Migrating 20220311020825-views-indexes.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220311020825-views-indexes.js in 0.021s"} +outline | {"label":"database","level":"info","message":"Migrating 20220319022812-events-indexes.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220319022812-events-indexes.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20220319060408-collection-create-permission.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220319060408-collection-create-permission.js in 0.015s"} +outline | {"label":"database","level":"info","message":"Migrating 20220328215615-add-shareId-to-search-queries.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220328215615-add-shareId-to-search-queries.js in 0.014s"} +outline | {"label":"database","level":"info","message":"Migrating 20220402032204-starred-collections.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220402032204-starred-collections.js in 0.03s"} +outline | {"label":"database","level":"info","message":"Migrating 20220409222213-user-flags.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220409222213-user-flags.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20220409225935-user-invited-by.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220409225935-user-invited-by.js in 0.017s"} +outline | {"label":"database","level":"info","message":"Migrating 20220413213537-add-inviteRequired-to-teams.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220413213537-add-inviteRequired-to-teams.js in 0.023s"} +outline | {"label":"database","level":"info","message":"Migrating 20220419052832-create-team-domains.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220419052832-create-team-domains.js in 0.024s"} +outline | {"label":"database","level":"info","message":"Migrating 20220421052253-create-file-operation-format.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220421052253-create-file-operation-format.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20220430043135-collection-sort-backfill.js…"} +outline | Backfilling collection sort… +outline | {"label":"database","level":"info","message":"Migrated 20220430043135-collection-sort-backfill.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20220521164111-create-webhook-subscription.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220521164111-create-webhook-subscription.js in 0.029s"} +outline | {"label":"database","level":"info","message":"Migrating 20220525054603-user-authentication-expires-at.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220525054603-user-authentication-expires-at.js in 0.017s"} +outline | {"label":"database","level":"info","message":"Migrating 20220606031139-create-webhook-delivieries.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220606031139-create-webhook-delivieries.js in 0.028s"} +outline | {"label":"database","level":"info","message":"Migrating 20220702132722-add-webhooks-deleted-at.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220702132722-add-webhooks-deleted-at.js in 0.016s"} +outline | {"label":"database","level":"info","message":"Migrating 20220719121200-create-subscriptions.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220719121200-create-subscriptions.js in 0.023s"} +outline | {"label":"database","level":"info","message":"Migrating 20220720221531-remove-deprecated-columns.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220720221531-remove-deprecated-columns.js in 0.027s"} +outline | {"label":"database","level":"info","message":"Migrating 20220722184916-remove-event-updatedat.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220722184916-remove-event-updatedat.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20220810185000-scope-provider-id-uniqueness-to-team.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220810185000-scope-provider-id-uniqueness-to-team.js in 0.017s"} +outline | {"label":"database","level":"info","message":"Migrating 20220812115059-scope-user-auth-provider-id-uniqueness-to-user.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220812115059-scope-user-auth-provider-id-uniqueness-to-user.js in 0.017s"} +outline | {"label":"database","level":"info","message":"Migrating 20220816070527-change-column-authentication-id-nullable.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220816070527-change-column-authentication-id-nullable.js in 0.015s"} +outline | {"label":"database","level":"info","message":"Migrating 20220816175234-user-email-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220816175234-user-email-index.js in 0.019s"} +outline | {"label":"database","level":"info","message":"Migrating 20220828144837-add-columns-to-notifications-for-tracking.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220828144837-add-columns-to-notifications-for-tracking.js in 0.033s"} +outline | {"label":"database","level":"info","message":"Migrating 20220830215146-add-shares-views.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220830215146-add-shares-views.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20220907132304-user-preferences.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220907132304-user-preferences.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20220907140227-team-preferences.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220907140227-team-preferences.js in 0.013s"} +outline | {"label":"database","level":"info","message":"Migrating 20220909203454-fix-notification-constraints.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220909203454-fix-notification-constraints.js in 0.053s"} +outline | {"label":"database","level":"info","message":"Migrating 20220922073737-webhook-signing-secret.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220922073737-webhook-signing-secret.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20220928030442-fix-avatar-urls.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20220928030442-fix-avatar-urls.js in 0.034s"} +outline | {"label":"database","level":"info","message":"Migrating 20221111171828-fix-user-constraints.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20221111171828-fix-user-constraints.js in 0.013s"} +outline | {"label":"database","level":"info","message":"Migrating 20221112152649-import-document-relationship.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20221112152649-import-document-relationship.js in 0.027s"} +outline | {"label":"database","level":"info","message":"Migrating 20221112162341-attachment-tracking.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20221112162341-attachment-tracking.js in 0.013s"} +outline | {"label":"database","level":"info","message":"Migrating 20221120151710-attachment-expiry.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20221120151710-attachment-expiry.js in 0.015s"} +outline | {"label":"database","level":"info","message":"Migrating 20221206163421-add-share-url-slug.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20221206163421-add-share-url-slug.js in 0.018s"} +outline | {"label":"database","level":"info","message":"Migrating 20221218013627-fix-webhook-subscription-constraints.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20221218013627-fix-webhook-subscription-constraints.js in 0.016s"} +outline | {"label":"database","level":"info","message":"Migrating 20221219013835-fix-integration-constraints.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20221219013835-fix-integration-constraints.js in 0.025s"} +outline | {"label":"database","level":"info","message":"Migrating 20221230234256-add-stars-cascade.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20221230234256-add-stars-cascade.js in 0.016s"} +outline | {"label":"database","level":"info","message":"Migrating 20230101144349-integration-indexes.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20230101144349-integration-indexes.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20230204191035-update-tsvector-trigger.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20230204191035-update-tsvector-trigger.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20230314013103-move-notification-settings.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20230314013103-move-notification-settings.js in 0.016s"} +outline | {"label":"database","level":"info","message":"Migrating 20230317144617-remove-user-username.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20230317144617-remove-user-username.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20230330181038-remove-column-userId-from-documents.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20230330181038-remove-column-userId-from-documents.js in 0.015s"} +outline | {"label":"database","level":"info","message":"Migrating 20230403120315-add-comment-to-notifications.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20230403120315-add-comment-to-notifications.js in 0.024s"} +outline | {"label":"database","level":"info","message":"Migrating 20230419124305-add-archivedat-col-to-notifications.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20230419124305-add-archivedat-col-to-notifications.js in 0.019s"} +outline | {"label":"database","level":"info","message":"Migrating 20230419132159-add-indexes-to-notifications.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20230419132159-add-indexes-to-notifications.js in 0.038s"} +outline | {"label":"database","level":"info","message":"Migrating 20230429005039-collection-admins.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20230429005039-collection-admins.js in 0.017s"} +outline | {"label":"database","level":"info","message":"Migrating 20230430213332-remove-notification-settings.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20230430213332-remove-notification-settings.js in 0.058s"} +outline | {"label":"database","level":"info","message":"Migrating 20230621004649-add-include-attachments-file-operation.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20230621004649-add-include-attachments-file-operation.js in 0.074s"} +outline | {"label":"database","level":"info","message":"Migrating 20230720002422-add-insights-control.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20230720002422-add-insights-control.js in 0.018s"} +outline | {"label":"database","level":"info","message":"Migrating 20230723231806-fix-file-operation-constraints.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20230723231806-fix-file-operation-constraints.js in 0.028s"} +outline | {"label":"database","level":"info","message":"Migrating 20230815063830-add-emoji-to-revisions.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20230815063830-add-emoji-to-revisions.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20230815063834-migrate-emoji-in-document-title.js…"} +outline | ⟐ injected env (0) · dotenvx@1.75.1 +outline | Backfill document emoji from title… page 0 +outline | (node:28) ExperimentalWarning: localStorage is not available because --localstorage-file was not provided. +outline | (Use `node --trace-warnings ...` to show where the warning was created) +outline | Backfill complete +outline | {"label":"database","level":"info","message":"Migrated 20230815063834-migrate-emoji-in-document-title.js in 2.884s"} +outline | {"label":"database","level":"info","message":"Migrating 20230827234031-migrate-emoji-in-revision-title.js…"} +outline | ⟐ injected env (0) · dotenvx@1.75.1 +outline | Backfill revision emoji from title… page 0 +outline | (node:41) ExperimentalWarning: localStorage is not available because --localstorage-file was not provided. +outline | (Use `node --trace-warnings ...` to show where the warning was created) +outline | Backfill complete +outline | {"label":"database","level":"info","message":"Migrated 20230827234031-migrate-emoji-in-revision-title.js in 2.769s"} +outline | {"label":"database","level":"info","message":"Migrating 20230920032853-add-key-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20230920032853-add-key-index.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20230921071140-rename-collection-groups.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20230921071140-rename-collection-groups.js in 0.036s"} +outline | {"label":"database","level":"info","message":"Migrating 20230922105047-rename-collection-users.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20230922105047-rename-collection-users.js in 0.042s"} +outline | {"label":"database","level":"info","message":"Migrating 20231001032754-file-operation-paranoid.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20231001032754-file-operation-paranoid.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20231101021239-share-domain.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20231101021239-share-domain.js in 0.017s"} +outline | {"label":"database","level":"info","message":"Migrating 20231103114720-add-column-index-to-user-permissions.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20231103114720-add-column-index-to-user-permissions.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20231111023920-add-source-metadata.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20231111023920-add-source-metadata.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20231118195149-add-content-to-documents.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20231118195149-add-content-to-documents.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20231120074257-add-column-id-to-user-permissions.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20231120074257-add-column-id-to-user-permissions.js in 0.023s"} +outline | {"label":"database","level":"info","message":"Migrating 20231120142213-add-column-id-to-group-permissions.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20231120142213-add-column-id-to-group-permissions.js in 0.017s"} +outline | {"label":"database","level":"info","message":"Migrating 20231123022323-add-suspended-at-teams.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20231123022323-add-suspended-at-teams.js in 0.009s"} +outline | {"label":"database","level":"info","message":"Migrating 20231129011114-cascade-delete.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20231129011114-cascade-delete.js in 0.015s"} +outline | {"label":"database","level":"info","message":"Migrating 20231206041706-search-query-score.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20231206041706-search-query-score.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20231212011038-search-query-answer.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20231212011038-search-query-answer.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20231227040129-update-tsvector-trigger.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20231227040129-update-tsvector-trigger.js in 0.009s"} +outline | {"label":"database","level":"info","message":"Migrating 20240113143315-user-permission-source-id.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240113143315-user-permission-source-id.js in 0.021s"} +outline | {"label":"database","level":"info","message":"Migrating 20240121172253-add-missing-cascades.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240121172253-add-missing-cascades.js in 0.028s"} +outline | {"label":"database","level":"info","message":"Migrating 20240203061519-integration-soft-delete.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240203061519-integration-soft-delete.js in 0.009s"} +outline | {"label":"database","level":"info","message":"Migrating 20240204171556-add-event-changeset.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240204171556-add-event-changeset.js in 0.009s"} +outline | {"label":"database","level":"info","message":"Migrating 20240204185157-team-last-active-at.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240204185157-team-last-active-at.js in 0.009s"} +outline | {"label":"database","level":"info","message":"Migrating 20240216182003-add-summary-to-documents.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240216182003-add-summary-to-documents.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20240229034214-search-query-relationship.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240229034214-search-query-relationship.js in 0.015s"} +outline | {"label":"database","level":"info","message":"Migrating 20240317171826-add-authentication-refresh-token.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240317171826-add-authentication-refresh-token.js in 0.009s"} +outline | {"label":"database","level":"info","message":"Migrating 20240319230356-fix-user-permissions-createdby-constraint.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240319230356-fix-user-permissions-createdby-constraint.js in 0.016s"} +outline | {"label":"database","level":"info","message":"Migrating 20240327015248-add-user-role.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240327015248-add-user-role.js in 0.014s"} +outline | {"label":"database","level":"info","message":"Migrating 20240327235446-role-non-nullable.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240327235446-role-non-nullable.js in 0.009s"} +outline | {"label":"database","level":"info","message":"Migrating 20240329012958-remove-old-role-columns.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240329012958-remove-old-role-columns.js in 0.013s"} +outline | {"label":"database","level":"info","message":"Migrating 20240413010743-add-options-to-file-operation.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240413010743-add-options-to-file-operation.js in 0.009s"} +outline | {"label":"database","level":"info","message":"Migrating 20240413042634-member-team-create.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240413042634-member-team-create.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20240524234042-add-content-to-collection.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240524234042-add-content-to-collection.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20240617030911-add-apikey-expiry.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240617030911-add-apikey-expiry.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20240617151506-add-icon-to-document.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240617151506-add-icon-to-document.js in 0.02s"} +outline | {"label":"database","level":"info","message":"Migrating 20240618201908-add-lastActiveAt-to-apikey.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240618201908-add-lastActiveAt-to-apikey.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20240625051656-remove-emoji-column.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240625051656-remove-emoji-column.js in 0.013s"} +outline | {"label":"database","level":"info","message":"Migrating 20240709031512-group-permission-source-id.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240709031512-group-permission-source-id.js in 0.021s"} +outline | {"label":"database","level":"info","message":"Migrating 20240717061527-add-column-archivedAt-collections.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240717061527-add-column-archivedAt-collections.js in 0.014s"} +outline | {"label":"database","level":"info","message":"Migrating 20240806080954-group-users-paranoid.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240806080954-group-users-paranoid.js in 0.009s"} +outline | {"label":"database","level":"info","message":"Migrating 20240809054702-add-column-archivedById-to-collections.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240809054702-add-column-archivedById-to-collections.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20240810080954-group-users-remove-deleted-at.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240810080954-group-users-remove-deleted-at.js in 0.018s"} +outline | {"label":"database","level":"info","message":"Migrating 20240821001616-add-notifications-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240821001616-add-notifications-index.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20240821002344-add-user-permission-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240821002344-add-user-permission-index.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20240821002502-add-user-authentication-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240821002502-add-user-authentication-index.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20240828081032-add-notifications-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240828081032-add-notifications-index.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20240912222438-add-unaccent-extension.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240912222438-add-unaccent-extension.js in 0.014s"} +outline | {"label":"database","level":"info","message":"Migrating 20240929194201-add-hash-to-api-key.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20240929194201-add-hash-to-api-key.js in 0.02s"} +outline | {"label":"database","level":"info","message":"Migrating 20240930113921-hash-api-keys.js…"} +outline | ⟐ injected env (0) · dotenvx@1.75.1 +outline | Backfill apiKey hash… page 0 +outline | (node:53) ExperimentalWarning: localStorage is not available because --localstorage-file was not provided. +outline | (Use `node --trace-warnings ...` to show where the warning was created) +outline | Backfill complete +outline | {"label":"database","level":"info","message":"Migrated 20240930113921-hash-api-keys.js in 2.769s"} +outline | {"label":"database","level":"info","message":"Migrating 20241013080608-create-reactions.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20241013080608-create-reactions.js in 0.022s"} +outline | {"label":"database","level":"info","message":"Migrating 20241030235556-event-auth-type.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20241030235556-event-auth-type.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20241105132600-add-allowIndexing-to-shares.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20241105132600-add-allowIndexing-to-shares.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20241105203523-add-user-timezone.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20241105203523-add-user-timezone.js in 0.009s"} +outline | {"label":"database","level":"info","message":"Migrating 20241127151705-attachment-tracking.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20241127151705-attachment-tracking.js in 0.013s"} +outline | {"label":"database","level":"info","message":"Migrating 20241219023150-group-external-id.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20241219023150-group-external-id.js in 0.016s"} +outline | {"label":"database","level":"info","message":"Migrating 20241231045203-add-attachments-teamId-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20241231045203-add-attachments-teamId-index.js in 0.013s"} +outline | {"label":"database","level":"info","message":"Migrating 20250125031823-add-api-key-scopes.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250125031823-add-api-key-scopes.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20250207120103-add-collectionId-to-subscriptions.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250207120103-add-collectionId-to-subscriptions.js in 0.014s"} +outline | {"label":"database","level":"info","message":"Migrating 20250217012609-document-title-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250217012609-document-title-index.js in 0.037s"} +outline | {"label":"database","level":"info","message":"Migrating 20250217230810-add-team-previous-subdomains.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250217230810-add-team-previous-subdomains.js in 0.015s"} +outline | {"label":"database","level":"info","message":"Migrating 20250223142558-cascade-team-domain.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250223142558-cascade-team-domain.js in 0.017s"} +outline | {"label":"database","level":"info","message":"Migrating 20250225153529-add-notification-membershipId.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250225153529-add-notification-membershipId.js in 0.009s"} +outline | {"label":"database","level":"info","message":"Migrating 20250301234423-add-name-to-revisions.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250301234423-add-name-to-revisions.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20250306181804-create-imports.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250306181804-create-imports.js in 0.027s"} +outline | {"label":"database","level":"info","message":"Migrating 20250310043011-create-import-tasks.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250310043011-create-import-tasks.js in 0.021s"} +outline | {"label":"database","level":"info","message":"Migrating 20250314105847-add-apiImportId-to-collections-and-documents.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250314105847-add-apiImportId-to-collections-and-documents.js in 0.021s"} +outline | {"label":"database","level":"info","message":"Migrating 20250327062414-resolve-collection-index-collisions.js…"} +outline | ⟐ injected env (0) · dotenvx@1.75.1 +outline | (node:65) ExperimentalWarning: localStorage is not available because --localstorage-file was not provided. +outline | (Use `node --trace-warnings ...` to show where the warning was created) +outline | {"label":"database","level":"info","message":"Migrated 20250327062414-resolve-collection-index-collisions.js in 2.82s"} +outline | {"label":"database","level":"info","message":"Migrating 20250327110739-add-error-to-imports.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250327110739-add-error-to-imports.js in 0.013s"} +outline | {"label":"database","level":"info","message":"Migrating 20250331231413-add-oauth-server-models.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250331231413-add-oauth-server-models.js in 0.039s"} +outline | {"label":"database","level":"info","message":"Migrating 20250409184249-add-issueSources-to-integrations.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250409184249-add-issueSources-to-integrations.js in 0.017s"} +outline | {"label":"database","level":"info","message":"Migrating 20250429130521-add-collection-commenting.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250429130521-add-collection-commenting.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20250515031645-add-revisions-deleted-at.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250515031645-add-revisions-deleted-at.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20250530235814-add-collaborator-ids-to-revisions.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250530235814-add-collaborator-ids-to-revisions.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20250531002344-add-description-to-teams.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250531002344-add-description-to-teams.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20250531003217-add-show-last-updated-to-shares.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250531003217-add-show-last-updated-to-shares.js in 0.02s"} +outline | {"label":"database","level":"info","message":"Migrating 20250601223331-migrate-backlink-to-relationship.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250601223331-migrate-backlink-to-relationship.js in 0.033s"} +outline | {"label":"database","level":"info","message":"Migrating 20250630100046-add-oauth-to-search-queries-source.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250630100046-add-oauth-to-search-queries-source.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20250630175759-add-collection-id-to-shares.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250630175759-add-collection-id-to-shares.js in 0.015s"} +outline | {"label":"database","level":"info","message":"Migrating 20250719063818-add-documentId-to-fileOperation.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250719063818-add-documentId-to-fileOperation.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20250810173939-remove-team-name-nullable.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250810173939-remove-team-name-nullable.js in 0.022s"} +outline | {"label":"database","level":"info","message":"Migrating 20250810210844-add-data-to-notifications.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250810210844-add-data-to-notifications.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20250826204500-modify-shares-unique-constraint-with-revoked-condition.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250826204500-modify-shares-unique-constraint-with-revoked-condition.js in 0.022s"} +outline | {"label":"database","level":"info","message":"Migrating 20250828094500-update-group-user-role.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250828094500-update-group-user-role.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20250830060500-add-expires-at-to-authentications.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250830060500-add-expires-at-to-authentications.js in 0.009s"} +outline | {"label":"database","level":"info","message":"Migrating 20250907220205-add-missing-indexes.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250907220205-add-missing-indexes.js in 0.021s"} +outline | {"label":"database","level":"info","message":"Migrating 20250913202342-add-source-metadata-to-collections.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250913202342-add-source-metadata-to-collections.js in 0.009s"} +outline | {"label":"database","level":"info","message":"Migrating 20250917124000-fix-shares-collection-cascade.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20250917124000-fix-shares-collection-cascade.js in 0.015s"} +outline | {"label":"database","level":"info","message":"Migrating 20251001133733-shares-show-toc.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20251001133733-shares-show-toc.js in 0.009s"} +outline | {"label":"database","level":"info","message":"Migrating 20251013204025-add-group-to-notifications.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20251013204025-add-group-to-notifications.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20251020204139-create-emojis.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20251020204139-create-emojis.js in 0.024s"} +outline | {"label":"database","level":"info","message":"Migrating 20251023031630-add-disable-mentions-to-groups.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20251023031630-add-disable-mentions-to-groups.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20251029200610-add-description-to-groups.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20251029200610-add-description-to-groups.js in 0.009s"} +outline | {"label":"database","level":"info","message":"Migrating 20251104013803-document-language.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20251104013803-document-language.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20251125012929-add-popularity-score-to-documents.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20251125012929-add-popularity-score-to-documents.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20251125012930-add-popularity-score-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20251125012930-add-popularity-score-index.js in 0.02s"} +outline | {"label":"database","level":"info","message":"Migrating 20251203124235-add-client-type-to-oauth-clients.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20251203124235-add-client-type-to-oauth-clients.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20251212232400-add-authentications-service-teamid-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20251212232400-add-authentications-service-teamid-index.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20251212232500-add-integrations-service-type-settings-indexes.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20251212232500-add-integrations-service-type-settings-indexes.js in 0.014s"} +outline | {"label":"database","level":"info","message":"Migrating 20251213004745-add-search-queries-source-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20251213004745-add-search-queries-source-index.js in 0.014s"} +outline | {"label":"database","level":"info","message":"Migrating 20251217161051-remove-unused-indexes.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20251217161051-remove-unused-indexes.js in 0.036s"} +outline | {"label":"database","level":"info","message":"Migrating 20251217204338-add-integration-performance-indexes.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20251217204338-add-integration-performance-indexes.js in 0.027s"} +outline | {"label":"database","level":"info","message":"Migrating 20251218223224-add-grant-id-to-oauth-models.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20251218223224-add-grant-id-to-oauth-models.js in 0.02s"} +outline | {"label":"database","level":"info","message":"Migrating 20260104155138-team-passkeys-enabled.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260104155138-team-passkeys-enabled.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20260104155139-create-user-passkeys.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260104155139-create-user-passkeys.js in 0.022s"} +outline | {"label":"database","level":"info","message":"Migrating 20260107213946-create-access-requests.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260107213946-create-access-requests.js in 0.033s"} +outline | {"label":"database","level":"info","message":"Migrating 20260107220000-add-access-request-to-notifications.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260107220000-add-access-request-to-notifications.js in 0.015s"} +outline | {"label":"database","level":"info","message":"Migrating 20260215215401-update-oauth-clients.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260215215401-update-oauth-clients.js in 0.022s"} +outline | {"label":"database","level":"info","message":"Migrating 20260221131935-add-client-credentials-to-authentications.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260221131935-add-client-credentials-to-authentications.js in 0.012s"} +outline | {"label":"database","level":"info","message":"Migrating 20260222000000-add-settings-to-authentication-providers.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260222000000-add-settings-to-authentication-providers.js in 0.009s"} +outline | {"label":"database","level":"info","message":"Migrating 20260222000001-create-external-groups.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260222000001-create-external-groups.js in 0.023s"} +outline | {"label":"database","level":"info","message":"Migrating 20260224021319-add-subscription-event-indexes.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260224021319-add-subscription-event-indexes.js in 0.026s"} +outline | {"label":"database","level":"info","message":"Migrating 20260314000000-add-team-flags.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260314000000-add-team-flags.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20260319221917-add-collection-template-management.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260319221917-add-collection-template-management.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20260320000000-add-team-guidance-mcp.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260320000000-add-team-guidance-mcp.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20260330000000-create-share-subscriptions.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260330000000-create-share-subscriptions.js in 0.021s"} +outline | {"label":"database","level":"info","message":"Migrating 20260331000000-add-allowSubscriptions-to-shares.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260331000000-add-allowSubscriptions-to-shares.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20260401000000-add-documentId-to-share-subscriptions.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260401000000-add-documentId-to-share-subscriptions.js in 0.017s"} +outline | {"label":"database","level":"info","message":"Migrating 20260411000000-add-title-and-iconUrl-to-shares.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260411000000-add-title-and-iconUrl-to-shares.js in 0.013s"} +outline | {"label":"database","level":"info","message":"Migrating 20260416000000-create-document-insights.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260416000000-create-document-insights.js in 0.023s"} +outline | {"label":"database","level":"info","message":"Migrating 20260419000000-add-period-to-document-insights.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260419000000-add-period-to-document-insights.js in 0.018s"} +outline | {"label":"database","level":"info","message":"Migrating 20260514000000-cascade-resolved-to-reply-comments.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260514000000-cascade-resolved-to-reply-comments.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20260515000000-make-imports-integration-nullable.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260515000000-make-imports-integration-nullable.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20260516000000-add-import-task-phase-and-import-scratch.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260516000000-add-import-task-phase-and-import-scratch.js in 0.018s"} +outline | {"label":"database","level":"info","message":"Migrating 20260526092836-add-document-foreign-key-indexes.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260526092836-add-document-foreign-key-indexes.js in 0.036s"} +outline | {"label":"database","level":"info","message":"Migrating 20260603000000-commenting-access-preference.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260603000000-commenting-access-preference.js in 0.011s"} +outline | {"label":"database","level":"info","message":"Migrating 20260604140753-increase-url-column-lengths.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260604140753-increase-url-column-lengths.js in 0.031s"} +outline | {"label":"database","level":"info","message":"Migrating 20260611000000-drop-team-collaborative-editing.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260611000000-drop-team-collaborative-editing.js in 0.014s"} +outline | {"label":"database","level":"info","message":"Migrating 20260619000000-add-duration-to-search-queries.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260619000000-add-duration-to-search-queries.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20260621000000-add-search-queries-user-created-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260621000000-add-search-queries-user-created-index.js in 0.029s"} +outline | {"label":"database","level":"info","message":"Migrating 20260714000000-add-mcp-to-search-queries-source.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260714000000-add-mcp-to-search-queries-source.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20260718134528-add-search-queries-created-at-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260718134528-add-search-queries-created-at-index.js in 0.015s"} +outline | {"label":"database","level":"info","message":"Migrating 20260803143858-add-documents-team-search-vector-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260803143858-add-documents-team-search-vector-index.js in 0.129s"} +outline | {"label":"database","level":"info","message":"Migrating 20260804130000-add-documents-active-popularity-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260804130000-add-documents-active-popularity-index.js in 0.016s"} +outline | {"label":"database","level":"info","message":"Migrating 20260805120000-add-revisions-source-metadata.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260805120000-add-revisions-source-metadata.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20260818120000-add-deletedById-to-documents.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260818120000-add-deletedById-to-documents.js in 0.046s"} +outline | {"label":"database","level":"info","message":"Migrating 20260821164024-add-preferences-to-documents.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260821164024-add-preferences-to-documents.js in 0.009s"} +outline | {"label":"database","level":"info","message":"Migrating 20260902125149-limit-document-search-trigger-columns.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260902125149-limit-document-search-trigger-columns.js in 0.01s"} +outline | {"label":"database","level":"info","message":"Migrating 20260903145856-add-document-createdbyid-index.js…"} +outline | {"label":"database","level":"info","message":"Migrated 20260903145856-add-document-createdbyid-index.js in 0.043s"} +outline | {"label":"lifecycle","level":"info","message":"\nIs your team enjoying Outline? Consider supporting future development by sponsoring the project:\n\nhttps://github.com/sponsors/outline\n"} +outline | {"label":"lifecycle","level":"info","message":"Note: Restricting process count to 1 due to use of collaborative service without REDIS_COLLABORATION_URL"} +outline | {"label":"lifecycle","level":"info","message":"Starting collaboration service"} +outline | {"label":"lifecycle","level":"info","message":"Starting websockets service"} +outline | {"label":"lifecycle","level":"info","message":"Starting worker service"} +outline | (node:77) ExperimentalWarning: localStorage is not available because --localstorage-file was not provided. +outline | (Use `node --trace-warnings ...` to show where the warning was created) +outline | {"label":"lifecycle","level":"info","message":"Starting web service"} +outline | {"label":"lifecycle","level":"info","message":"Listening on http://localhost:3000 / https://kb.gate.invalid"} +outline | {"attempt":0,"event":{"actorId":null,"authType":null,"changes":{"attributes":{"avatarUrl":"[…]","name":"[…]","subdomain":"[…]"},"previous":{}},"collectionId":null,"createdAt":"2026-09-30T20:22:16.025Z","data":null,"documentId":null,"id":"6928c89f-4ea2-44ae-9e90-6212d052cb51","ip":"172.18.0.1","modelId":"b837af95-5399-41f1-a0e8-fcd8c221e74f","name":"teams.create","teamId":"b837af95-5399-41f1-a0e8-fcd8c221e74f","userId":null},"label":"worker","level":"info","message":"Processing teams.create"} +outline | {"attempt":0,"event":{"actorId":"f0253c8e-78fa-4736-9cd9-9e6732e7c5eb","authType":null,"changes":{"attributes":{"email":"[…]","name":"[…]","role":"[…]","teamId":"[…]"},"previous":{}},"collectionId":null,"createdAt":"2026-09-30T20:22:16.040Z","data":null,"documentId":null,"id":"19ab7c9f-62a8-442b-9ee4-d97c33ec47e0","ip":"172.18.0.1","modelId":"f0253c8e-78fa-4736-9cd9-9e6732e7c5eb","name":"users.create","teamId":"b837af95-5399-41f1-a0e8-fcd8c221e74f","userId":"f0253c8e-78fa-4736-9cd9-9e6732e7c5eb"},"label":"worker","level":"info","message":"Processing users.create"} +outline | {"attempt":0,"event":{"actorId":"f0253c8e-78fa-4736-9cd9-9e6732e7c5eb","authType":null,"changes":null,"collectionId":null,"createdAt":"2026-09-30T20:22:16.052Z","data":{"name":"Drill","service":"email"},"documentId":null,"id":"54b73299-e5b1-488e-b163-6a6a5f1b5296","ip":"172.18.0.1","modelId":null,"name":"users.signin","teamId":"b837af95-5399-41f1-a0e8-fcd8c221e74f","userId":"f0253c8e-78fa-4736-9cd9-9e6732e7c5eb"},"label":"worker","level":"info","message":"Processing users.signin"} +outline | {"event":{"actorId":null,"authType":null,"changes":{"attributes":{"avatarUrl":"[…]","name":"[…]","subdomain":"[…]"},"previous":{}},"collectionId":null,"createdAt":"2026-09-30T20:22:16.025Z","data":null,"documentId":null,"id":"6928c89f-4ea2-44ae-9e90-6212d052cb51","ip":"172.18.0.1","modelId":"b837af95-5399-41f1-a0e8-fcd8c221e74f","name":"teams.create","teamId":"b837af95-5399-41f1-a0e8-fcd8c221e74f","userId":null},"label":"worker","level":"info","message":"AvatarProcessor running teams.create"} +outline | {"event":{"actorId":"f0253c8e-78fa-4736-9cd9-9e6732e7c5eb","authType":null,"changes":{"attributes":{"email":"[…]","name":"[…]","role":"[…]","teamId":"[…]"},"previous":{}},"collectionId":null,"createdAt":"2026-09-30T20:22:16.040Z","data":null,"documentId":null,"id":"19ab7c9f-62a8-442b-9ee4-d97c33ec47e0","ip":"172.18.0.1","modelId":"f0253c8e-78fa-4736-9cd9-9e6732e7c5eb","name":"users.create","teamId":"b837af95-5399-41f1-a0e8-fcd8c221e74f","userId":"f0253c8e-78fa-4736-9cd9-9e6732e7c5eb"},"label":"worker","level":"info","message":"AvatarProcessor running users.create"} +outline | {"event":{"actorId":"f0253c8e-78fa-4736-9cd9-9e6732e7c5eb","authType":null,"changes":{"attributes":{"email":"[…]","name":"[…]","role":"[…]","teamId":"[…]"},"previous":{}},"collectionId":null,"createdAt":"2026-09-30T20:22:16.040Z","data":null,"documentId":null,"id":"19ab7c9f-62a8-442b-9ee4-d97c33ec47e0","ip":"172.18.0.1","modelId":"f0253c8e-78fa-4736-9cd9-9e6732e7c5eb","name":"users.create","teamId":"b837af95-5399-41f1-a0e8-fcd8c221e74f","userId":"f0253c8e-78fa-4736-9cd9-9e6732e7c5eb"},"label":"worker","level":"info","message":"UserCreatedProcessor running users.create"} +outline | {"label":"email","level":"info","message":"Email WelcomeEmail not sent due to missing SMTP_FROM_EMAIL configuration"} diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/bench-outline-attempt-fixture-fault/evidence/RT-outline/run.log b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench-outline-attempt-fixture-fault/evidence/RT-outline/run.log new file mode 100644 index 00000000..fa727949 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench-outline-attempt-fixture-fault/evidence/RT-outline/run.log @@ -0,0 +1,6 @@ +[20:20:51] scratch drive folders cleared before FROM (R-656): none existed +[20:20:51] RT-outline: deploying outline at FROM {'outline': 'outlinewiki/outline:1.10.1', 'outline-postgres': 'postgres:18-alpine', 'outline-redis': 'redis:7-alpine@sha256:c35af3bbcef51a62c8bae5a9a563c6f1b60d7ebaea4cb5a3ccbcc157580ae098'} +[20:22:15] FROM settled=True in 31.2s :: {"outline": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "outline-postgres": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "outline-redis": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}} +[20:22:15] fixture: the BOX walk's own (Outline), through upgrade_boxport +[20:22:16] outline: installation.create http=302 +[20:22:16] INCONCLUSIVE — no non-browser seed route. Nothing was planted by hand. \ No newline at end of file diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/bench-outline-attempt-fixture-fault/evidence/RT-outline/verdict.json b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench-outline-attempt-fixture-fault/evidence/RT-outline/verdict.json new file mode 100644 index 00000000..d00d0d89 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench-outline-attempt-fixture-fault/evidence/RT-outline/verdict.json @@ -0,0 +1,31 @@ +{ + "harness_version": 4, + "edge": "RT-outline", + "app": "outline", + "note": "re-test of the same tag at a new digest (decision 52): outline-redis sha256:c35af3bbcef5 -> sha256:858f009f9709", + "from": { + "outline": "outlinewiki/outline:1.10.1", + "outline-postgres": "postgres:18-alpine", + "outline-redis": "redis:7-alpine@sha256:c35af3bbcef51a62c8bae5a9a563c6f1b60d7ebaea4cb5a3ccbcc157580ae098" + }, + "to": { + "outline": "outlinewiki/outline:1.10.1", + "outline-postgres": "postgres:18-alpine", + "outline-redis": "redis:7-alpine@sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499" + }, + "verdict": "inconclusive", + "seed_read_before": false, + "seed_read_after": false, + "healthy_after": false, + "migration_observed": null, + "abort": "not-attempted", + "abort_detail": "no non-browser seed route \u2014 tried: no csrfToken cookie from GET /home", + "engine_state_after": null, + "memory": null, + "marks": [], + "duration_s": 84.4, + "measured_at": "2026-09-30T20:22:16Z", + "evidence": "evidence/RT-outline", + "scratch_cleared": [], + "total_s": 84.4 +} \ No newline at end of file diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/RT-docmost.log b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/RT-docmost.log new file mode 100644 index 00000000..303dca96 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/RT-docmost.log @@ -0,0 +1,145 @@ +[20:24:34] scratch drive folders cleared before FROM (R-656): none existed +[20:24:34] RT-docmost: deploying docmost at FROM {'docmost': 'docmost/docmost:0.96.0', 'docmost-postgres': 'postgres:18-alpine', 'docmost-redis': 'redis:7-alpine@sha256:c35af3bbcef51a62c8bae5a9a563c6f1b60d7ebaea4cb5a3ccbcc157580ae098'} +[20:25:58] FROM settled=True in 31.1s :: {"docmost": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "docmost-postgres": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "docmost-redis": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}} +[20:25:58] docmost: /api/auth/setup http=200 rc=0 +[20:25:58] docmost: login as the seeded user http=200 ok=True +[20:25:58] C1 (seed reads back BEFORE): True +[20:25:59] RT-docmost: swapping to TO {'docmost': 'docmost/docmost:0.96.0', 'docmost-postgres': 'postgres:18-alpine', 'docmost-redis': 'redis:7-alpine@sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499'} +[20:26:17] TO up -d rc=0 +[20:26:48] TO settled=True in 31.1s :: {"docmost": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "docmost-postgres": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "docmost-redis": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}} +[20:26:49] engine state docmost-postgres: 18 +[20:26:49] migration lines observed: 1 +[20:26:49] docmost: login as the seeded user http=200 ok=True +[20:26:49] RESULT (seed reads back AFTER): True +[20:26:49] memory watch: 600s, 4 callers on 1 path(s) at 172.18.0.3:3000 +[20:27:05] + 15s docmost=426M/512M peak=467M kills=0 rs=0 docmost-postgres=111M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=300 +[20:27:20] + 30s docmost=432M/512M peak=467M kills=0 rs=0 docmost-postgres=112M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=600 +[20:27:35] + 46s docmost=438M/512M peak=467M kills=0 rs=0 docmost-postgres=105M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=896 +[20:27:50] + 61s docmost=435M/512M peak=467M kills=0 rs=0 docmost-postgres=90M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=1196 +[20:28:05] + 76s docmost=423M/512M peak=467M kills=0 rs=0 docmost-postgres=76M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=1496 +[20:28:20] + 91s docmost=423M/512M peak=467M kills=0 rs=0 docmost-postgres=72M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=1796 +[20:28:36] + 106s docmost=420M/512M peak=467M kills=0 rs=0 docmost-postgres=72M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=2096 +[20:28:51] + 121s docmost=422M/512M peak=467M kills=0 rs=0 docmost-postgres=72M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=2392 +[20:29:06] + 136s docmost=423M/512M peak=467M kills=0 rs=0 docmost-postgres=72M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=2692 +[20:29:21] + 152s docmost=423M/512M peak=467M kills=0 rs=0 docmost-postgres=72M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=2992 +[20:29:36] + 167s docmost=422M/512M peak=467M kills=0 rs=0 docmost-postgres=72M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=3288 +[20:29:51] + 182s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=72M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=3588 +[20:30:07] + 197s docmost=420M/512M peak=467M kills=0 rs=0 docmost-postgres=72M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=3888 +[20:30:22] + 212s docmost=419M/512M peak=467M kills=0 rs=0 docmost-postgres=73M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=4184 +[20:30:37] + 227s docmost=418M/512M peak=467M kills=0 rs=0 docmost-postgres=73M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=4484 +[20:30:52] + 243s docmost=418M/512M peak=467M kills=0 rs=0 docmost-postgres=73M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=4784 +[20:31:07] + 258s docmost=416M/512M peak=467M kills=0 rs=0 docmost-postgres=107M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=5084 +[20:31:22] + 273s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=106M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=5380 +[20:31:38] + 288s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=106M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=5680 +[20:31:53] + 303s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=106M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=5980 +[20:32:08] + 318s docmost=416M/512M peak=467M kills=0 rs=0 docmost-postgres=106M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=6276 +[20:32:23] + 334s docmost=416M/512M peak=467M kills=0 rs=0 docmost-postgres=101M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=6576 +[20:32:38] + 349s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=101M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=6876 +[20:32:53] + 364s docmost=416M/512M peak=467M kills=0 rs=0 docmost-postgres=101M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=7176 +[20:33:08] + 379s docmost=416M/512M peak=467M kills=0 rs=0 docmost-postgres=101M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=7472 +[20:33:24] + 394s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=101M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=7772 +[20:33:39] + 409s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=101M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=8072 +[20:33:54] + 424s docmost=415M/512M peak=467M kills=0 rs=0 docmost-postgres=98M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=8372 +[20:34:09] + 440s docmost=416M/512M peak=467M kills=0 rs=0 docmost-postgres=94M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=8672 +[20:34:24] + 455s docmost=415M/512M peak=467M kills=0 rs=0 docmost-postgres=93M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=8968 +[20:34:39] + 470s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=93M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=9268 +[20:34:55] + 485s docmost=414M/512M peak=467M kills=0 rs=0 docmost-postgres=93M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=8M kills=0 rs=0 reqs=9568 +[20:35:10] + 500s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=93M/256M peak=125M kills=0 rs=0 docmost-redis=6M/128M peak=8M kills=0 rs=0 reqs=9868 +[20:35:25] + 516s docmost=415M/512M peak=467M kills=0 rs=0 docmost-postgres=93M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=8M kills=0 rs=0 reqs=10166 +[20:35:40] + 531s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=93M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=8M kills=0 rs=0 reqs=10464 +[20:35:55] + 546s docmost=414M/512M peak=467M kills=0 rs=0 docmost-postgres=92M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=8M kills=0 rs=0 reqs=10764 +[20:36:10] + 561s docmost=414M/512M peak=467M kills=0 rs=0 docmost-postgres=92M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=8M kills=0 rs=0 reqs=11064 +[20:36:26] + 576s docmost=413M/512M peak=467M kills=0 rs=0 docmost-postgres=92M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=8M kills=0 rs=0 reqs=11360 +[20:36:41] + 591s docmost=414M/512M peak=467M kills=0 rs=0 docmost-postgres=92M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=8M kills=0 rs=0 reqs=11660 +[20:36:56] + 606s docmost=415M/512M peak=467M kills=0 rs=0 docmost-postgres=89M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=8M kills=0 rs=0 reqs=11960 +[20:36:56] memory watch: killed=False tight=['docmost'] requests=11960 codes={'200': 11960} +[20:36:56] RT-docmost: ABORT — putting the FROM images back +[20:37:40] docmost: login as the seeded user http=200 ok=True +[20:37:40] ABORT: app came back in 31.1s; data present=True +{ + "harness_version": 4, + "edge": "RT-docmost", + "app": "docmost", + "note": "re-test of the same tag at a new digest (decision 52): docmost-redis sha256:c35af3bbcef5 -> sha256:858f009f9709", + "from": { + "docmost": "docmost/docmost:0.96.0", + "docmost-postgres": "postgres:18-alpine", + "docmost-redis": "redis:7-alpine@sha256:c35af3bbcef51a62c8bae5a9a563c6f1b60d7ebaea4cb5a3ccbcc157580ae098" + }, + "to": { + "docmost": "docmost/docmost:0.96.0", + "docmost-postgres": "postgres:18-alpine", + "docmost-redis": "redis:7-alpine@sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499" + }, + "verdict": "proven", + "seed_read_before": true, + "seed_read_after": true, + "healthy_after": true, + "migration_observed": "\u001b[2Kdocmost | {\"level\":\"info\",\"time\":\"2026-09-30T20:26:24.623Z\",\"pid\":45,\"hostname\":\"15f6b2bd7ca1\",\"context\":\"DatabaseMigrationService\",\"msg\":\"No pending database migrations\"}", + "abort": "starts-and-serves", + "abort_detail": null, + "engine_state_after": { + "docmost-postgres": { + "image": "postgres:18-alpine", + "probe": "datadir major version", + "answer": "18", + "probe_rc": 0 + } + }, + "memory": { + "soak_s": 606.7, + "requested_s": 600, + "requests": 11960, + "codes": { + "200": 11960 + }, + "first_kill": null, + "containers": { + "docmost": { + "limit": 536870912, + "peak": 490369024, + "peak_pct": 0.913, + "anon_peak_sampled": 432144384, + "anon_peak_pct": 0.805, + "oom_kills": 0, + "restarts": 0, + "oomkilled_flag": false, + "measured": true + }, + "docmost-postgres": { + "limit": 268435456, + "peak": 131739648, + "peak_pct": 0.491, + "anon_peak_sampled": 11714560, + "anon_peak_pct": 0.044, + "oom_kills": 0, + "restarts": 0, + "oomkilled_flag": false, + "measured": true + }, + "docmost-redis": { + "limit": 134217728, + "peak": 8601600, + "peak_pct": 0.064, + "anon_peak_sampled": 4161536, + "anon_peak_pct": 0.031, + "oom_kills": 0, + "restarts": 0, + "oomkilled_flag": false, + "measured": true + } + }, + "unmeasured": [], + "load": "reached" + }, + "marks": [ + "memory_tight" + ], + "duration_s": 31.1, + "measured_at": "2026-09-30T20:37:40Z", + "evidence": "evidence/RT-docmost", + "scratch_cleared": [], + "files_changed": [], + "files_changed_detail": [], + "total_s": 785.1 +} diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/abort-states.json b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/abort-states.json new file mode 100644 index 00000000..c6bae4a5 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/abort-states.json @@ -0,0 +1,20 @@ +{ + "docmost": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + }, + "docmost-postgres": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + }, + "docmost-redis": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + } +} \ No newline at end of file diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/compose-final.log b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/compose-final.log new file mode 100644 index 00000000..708c2009 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/compose-final.log @@ -0,0 +1,88 @@ +docmost-redis | 1:C 30 Sep 2026 22:36:57.782 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. +docmost-redis | 1:C 30 Sep 2026 22:36:57.783 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo +docmost-redis | 1:C 30 Sep 2026 22:36:57.783 * Redis version=7.4.0, bits=64, commit=00000000, modified=0, pid=1, just started +docmost-redis | 1:C 30 Sep 2026 22:36:57.783 * Configuration loaded +docmost-redis | 1:M 30 Sep 2026 22:36:57.783 * monotonic clock: POSIX clock_gettime +docmost-redis | 1:M 30 Sep 2026 22:36:57.786 * Running mode=standalone, port=6379. +docmost-redis | 1:M 30 Sep 2026 22:36:57.787 * Server initialized +docmost-redis | 1:M 30 Sep 2026 22:36:57.788 * Reading RDB base file on AOF loading... +docmost-redis | 1:M 30 Sep 2026 22:36:57.788 * Loading RDB produced by version 7.4.0 +docmost-redis | 1:M 30 Sep 2026 22:36:57.788 * RDB age 701 seconds +docmost-redis | 1:M 30 Sep 2026 22:36:57.788 * RDB memory usage when created 0.90 Mb +docmost-redis | 1:M 30 Sep 2026 22:36:57.788 * RDB is base AOF +docmost-redis | 1:M 30 Sep 2026 22:36:57.788 * Done loading RDB, keys loaded: 0, keys expired: 0. +docmost-redis | 1:M 30 Sep 2026 22:36:57.788 * DB loaded from base file appendonly.aof.1.base.rdb: 0.001 seconds +docmost-redis | 1:M 30 Sep 2026 22:36:57.798 * DB loaded from incr file appendonly.aof.1.incr.aof: 0.009 seconds +docmost-redis | 1:M 30 Sep 2026 22:36:57.798 * DB loaded from append only file: 0.011 seconds +docmost-redis | 1:M 30 Sep 2026 22:36:57.798 * Opening AOF incr file appendonly.aof.1.incr.aof on server start +docmost-redis | 1:M 30 Sep 2026 22:36:57.798 * Ready to accept connections tcp +docmost-postgres | The files belonging to this database system will be owned by user "postgres". +docmost-postgres | This user must also own the server process. +docmost-postgres | +docmost-postgres | The database cluster will be initialized with locale "en_US.utf8". +docmost-postgres | The default database encoding has accordingly been set to "UTF8". +docmost-postgres | The default text search configuration will be set to "english". +docmost-postgres | +docmost-postgres | Data page checksums are enabled. +docmost-postgres | +docmost-postgres | fixing permissions on existing directory /var/lib/postgresql/18/docker ... ok +docmost-postgres | creating subdirectories ... ok +docmost-postgres | selecting dynamic shared memory implementation ... posix +docmost-postgres | selecting default "max_connections" ... 100 +docmost-postgres | selecting default "shared_buffers" ... 128MB +docmost-postgres | selecting default time zone ... Europe/Budapest +docmost-postgres | creating configuration files ... ok +docmost-postgres | running bootstrap script ... ok +docmost-postgres | sh: locale: not found +docmost-postgres | 2026-09-30 22:25:16.617 CEST [40] WARNING: no usable system locales were found +docmost-postgres | performing post-bootstrap initialization ... ok +docmost-postgres | syncing data to disk ... ok +docmost-postgres | +docmost-postgres | +docmost-postgres | Success. You can now start the database server using: +docmost-postgres | +docmost-postgres | pg_ctl -D /var/lib/postgresql/18/docker -l logfile start +docmost-postgres | +docmost-postgres | initdb: warning: enabling "trust" authentication for local connections +docmost-postgres | initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. +docmost-postgres | waiting for server to start....2026-09-30 22:25:17.456 CEST [46] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-musl, compiled by gcc (Alpine 15.2.0) 15.2.0, 64-bit +docmost-postgres | 2026-09-30 22:25:17.460 CEST [46] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" +docmost-postgres | 2026-09-30 22:25:17.479 CEST [52] LOG: database system was shut down at 2026-09-30 22:25:17 CEST +docmost-postgres | 2026-09-30 22:25:17.488 CEST [46] LOG: database system is ready to accept connections +docmost-postgres | done +docmost-postgres | server started +docmost-postgres | CREATE DATABASE +docmost-postgres | +docmost-postgres | +docmost-postgres | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/* +docmost-postgres | +docmost-postgres | waiting for server to shut down....2026-09-30 22:25:17.648 CEST [46] LOG: received fast shutdown request +docmost-postgres | 2026-09-30 22:25:17.654 CEST [46] LOG: aborting any active transactions +docmost-postgres | 2026-09-30 22:25:17.657 CEST [46] LOG: background worker "logical replication launcher" (PID 55) exited with exit code 1 +docmost-postgres | 2026-09-30 22:25:17.659 CEST [50] LOG: shutting down +docmost-postgres | 2026-09-30 22:25:17.664 CEST [50] LOG: checkpoint starting: shutdown immediate +docmost-postgres | 2026-09-30 22:25:17.752 CEST [50] LOG: checkpoint complete: wrote 943 buffers (5.8%), wrote 3 SLRU buffers; 0 WAL file(s) added, 0 removed, 0 recycled; write=0.034 s, sync=0.033 s, total=0.093 s; sync files=303, longest=0.004 s, average=0.001 s; distance=4362 kB, estimate=4362 kB; lsn=0/1BA8858, redo lsn=0/1BA8858 +docmost-postgres | 2026-09-30 22:25:17.776 CEST [46] LOG: database system is shut down +docmost-postgres | done +docmost-postgres | server stopped +docmost-postgres | +docmost-postgres | PostgreSQL init process complete; ready for start up. +docmost-postgres | +docmost-postgres | 2026-09-30 22:25:17.897 CEST [1] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-musl, compiled by gcc (Alpine 15.2.0) 15.2.0, 64-bit +docmost-postgres | 2026-09-30 22:25:17.897 CEST [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 +docmost-postgres | 2026-09-30 22:25:17.897 CEST [1] LOG: listening on IPv6 address "::", port 5432 +docmost-postgres | 2026-09-30 22:25:17.905 CEST [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" +docmost-postgres | 2026-09-30 22:25:17.918 CEST [68] LOG: database system was shut down at 2026-09-30 22:25:17 CEST +docmost-postgres | 2026-09-30 22:25:17.929 CEST [1] LOG: database system is ready to accept connections +docmost-postgres | 2026-09-30 22:30:18.018 CEST [66] LOG: checkpoint starting: time +docmost-postgres | 2026-09-30 22:31:07.162 CEST [66] LOG: checkpoint complete: wrote 489 buffers (3.0%), wrote 3 SLRU buffers; 1 WAL file(s) added, 0 removed, 0 recycled; write=49.046 s, sync=0.050 s, total=49.145 s; sync files=422, longest=0.003 s, average=0.001 s; distance=3849 kB, estimate=3849 kB; lsn=0/1F6B0D8, redo lsn=0/1F6B048 +docmost | $ pnpm --filter ./apps/server run start:prod +docmost | $ cross-env NODE_ENV=production node dist/main +docmost | (node:45) ExperimentalWarning: localStorage is not available because --localstorage-file was not provided. +docmost | (Use `node --trace-warnings ...` to show where the warning was created) +docmost | {"level":"info","time":"2026-09-30T20:37:15.593Z","pid":45,"hostname":"cc3f10dc44d0","context":"RedisModule","msg":"default: the connection was successfully established"} +docmost | {"level":"info","time":"2026-09-30T20:37:15.900Z","pid":45,"hostname":"cc3f10dc44d0","context":"DatabaseModule","msg":"Establishing database connection"} +docmost | {"level":"info","time":"2026-09-30T20:37:15.935Z","pid":45,"hostname":"cc3f10dc44d0","context":"DatabaseModule","msg":"Database connection successful"} +docmost | {"level":"info","time":"2026-09-30T20:37:16.095Z","pid":45,"hostname":"cc3f10dc44d0","context":"DatabaseMigrationService","msg":"No pending database migrations"} +docmost | {"level":"info","time":"2026-09-30T20:37:16.128Z","pid":45,"hostname":"cc3f10dc44d0","context":"NestApplication","msg":"Nest application successfully started"} +docmost | {"level":"info","time":"2026-09-30T20:37:16.141Z","pid":45,"hostname":"cc3f10dc44d0","context":"NestApplication","msg":"Listening on http://127.0.0.1:3000 / https://docs.gate.invalid"} diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/engine-state.json b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/engine-state.json new file mode 100644 index 00000000..6c38c07a --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/engine-state.json @@ -0,0 +1,8 @@ +{ + "docmost-postgres": { + "image": "postgres:18-alpine", + "probe": "datadir major version", + "answer": "18", + "probe_rc": 0 + } +} \ No newline at end of file diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/files-after-detail.json b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/files-after-detail.json new file mode 100644 index 00000000..9e26dfee --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/files-after-detail.json @@ -0,0 +1 @@ +{} \ No newline at end of file diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/files-after.json b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/files-after.json new file mode 100644 index 00000000..9e26dfee --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/files-after.json @@ -0,0 +1 @@ +{} \ No newline at end of file diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/files-before-detail.json b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/files-before-detail.json new file mode 100644 index 00000000..9e26dfee --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/files-before-detail.json @@ -0,0 +1 @@ +{} \ No newline at end of file diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/files-before.json b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/files-before.json new file mode 100644 index 00000000..9e26dfee --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/files-before.json @@ -0,0 +1 @@ +{} \ No newline at end of file diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/memory-samples.json b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/memory-samples.json new file mode 100644 index 00000000..ff0b4e57 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/memory-samples.json @@ -0,0 +1,1562 @@ +[ + { + "t": 15.2, + "containers": { + "docmost": { + "limit": 536870912, + "current": 447430656, + "peak": 490369024, + "anon": 420286464, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 117428224, + "peak": 131739648, + "anon": 11460608, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5132288, + "peak": 7704576, + "anon": 4108288, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 300 + }, + { + "t": 30.3, + "containers": { + "docmost": { + "limit": 536870912, + "current": 453046272, + "peak": 490369024, + "anon": 426172416, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 117616640, + "peak": 131739648, + "anon": 11460608, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5210112, + "peak": 7704576, + "anon": 4132864, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 600 + }, + { + "t": 45.5, + "containers": { + "docmost": { + "limit": 536870912, + "current": 459845632, + "peak": 490369024, + "anon": 432144384, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 111009792, + "peak": 131739648, + "anon": 11460608, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5476352, + "peak": 7704576, + "anon": 4136960, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 896 + }, + { + "t": 60.6, + "containers": { + "docmost": { + "limit": 536870912, + "current": 457158656, + "peak": 490369024, + "anon": 430436352, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 94576640, + "peak": 131739648, + "anon": 11460608, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5242880, + "peak": 8122368, + "anon": 4153344, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 1196 + }, + { + "t": 75.8, + "containers": { + "docmost": { + "limit": 536870912, + "current": 443904000, + "peak": 490369024, + "anon": 417292288, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 80084992, + "peak": 131739648, + "anon": 11460608, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5545984, + "peak": 8384512, + "anon": 4161536, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 1496 + }, + { + "t": 91.0, + "containers": { + "docmost": { + "limit": 536870912, + "current": 444456960, + "peak": 490369024, + "anon": 417353728, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 76513280, + "peak": 131739648, + "anon": 11460608, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5255168, + "peak": 8384512, + "anon": 4128768, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 1796 + }, + { + "t": 106.1, + "containers": { + "docmost": { + "limit": 536870912, + "current": 440696832, + "peak": 490369024, + "anon": 414162944, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 76529664, + "peak": 131739648, + "anon": 11460608, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5230592, + "peak": 8384512, + "anon": 4112384, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 2096 + }, + { + "t": 121.3, + "containers": { + "docmost": { + "limit": 536870912, + "current": 443330560, + "peak": 490369024, + "anon": 416972800, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 76529664, + "peak": 131739648, + "anon": 11460608, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 4993024, + "peak": 8384512, + "anon": 4108288, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 2392 + }, + { + "t": 136.4, + "containers": { + "docmost": { + "limit": 536870912, + "current": 443867136, + "peak": 490369024, + "anon": 417107968, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 76533760, + "peak": 131739648, + "anon": 11460608, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5238784, + "peak": 8384512, + "anon": 4108288, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 2692 + }, + { + "t": 151.6, + "containers": { + "docmost": { + "limit": 536870912, + "current": 443686912, + "peak": 490369024, + "anon": 417353728, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 76529664, + "peak": 131739648, + "anon": 11460608, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5234688, + "peak": 8384512, + "anon": 4108288, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 2992 + }, + { + "t": 166.8, + "containers": { + "docmost": { + "limit": 536870912, + "current": 442757120, + "peak": 490369024, + "anon": 416444416, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 76537856, + "peak": 131739648, + "anon": 11460608, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5251072, + "peak": 8384512, + "anon": 4108288, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 3288 + }, + { + "t": 181.9, + "containers": { + "docmost": { + "limit": 536870912, + "current": 438022144, + "peak": 490369024, + "anon": 410923008, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 76537856, + "peak": 131739648, + "anon": 11468800, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5246976, + "peak": 8384512, + "anon": 4108288, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 3588 + }, + { + "t": 197.1, + "containers": { + "docmost": { + "limit": 536870912, + "current": 440496128, + "peak": 490369024, + "anon": 413069312, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 76537856, + "peak": 131739648, + "anon": 11468800, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5267456, + "peak": 8384512, + "anon": 4116480, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 3888 + }, + { + "t": 212.2, + "containers": { + "docmost": { + "limit": 536870912, + "current": 439463936, + "peak": 490369024, + "anon": 412565504, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 77230080, + "peak": 131739648, + "anon": 11571200, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5246976, + "peak": 8384512, + "anon": 4096000, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 4184 + }, + { + "t": 227.4, + "containers": { + "docmost": { + "limit": 536870912, + "current": 438374400, + "peak": 490369024, + "anon": 412237824, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 76959744, + "peak": 131739648, + "anon": 11575296, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5267456, + "peak": 8384512, + "anon": 4108288, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 4484 + }, + { + "t": 242.6, + "containers": { + "docmost": { + "limit": 536870912, + "current": 439144448, + "peak": 490369024, + "anon": 412303360, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 77352960, + "peak": 131739648, + "anon": 11591680, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5038080, + "peak": 8384512, + "anon": 4124672, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 4784 + }, + { + "t": 257.7, + "containers": { + "docmost": { + "limit": 536870912, + "current": 437198848, + "peak": 490369024, + "anon": 410845184, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 112197632, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5619712, + "peak": 8384512, + "anon": 4120576, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 5084 + }, + { + "t": 272.9, + "containers": { + "docmost": { + "limit": 536870912, + "current": 437747712, + "peak": 490369024, + "anon": 411471872, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 111935488, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5517312, + "peak": 8384512, + "anon": 4112384, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 5380 + }, + { + "t": 288.0, + "containers": { + "docmost": { + "limit": 536870912, + "current": 438046720, + "peak": 490369024, + "anon": 411668480, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 111865856, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5046272, + "peak": 8384512, + "anon": 4116480, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 5680 + }, + { + "t": 303.2, + "containers": { + "docmost": { + "limit": 536870912, + "current": 437334016, + "peak": 490369024, + "anon": 410935296, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 111865856, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5308416, + "peak": 8384512, + "anon": 4112384, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 5980 + }, + { + "t": 318.3, + "containers": { + "docmost": { + "limit": 536870912, + "current": 436961280, + "peak": 490369024, + "anon": 410849280, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 112005120, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5316608, + "peak": 8384512, + "anon": 4112384, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 6276 + }, + { + "t": 333.5, + "containers": { + "docmost": { + "limit": 536870912, + "current": 436932608, + "peak": 490369024, + "anon": 410857472, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 106471424, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5320704, + "peak": 8384512, + "anon": 4112384, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 6576 + }, + { + "t": 348.7, + "containers": { + "docmost": { + "limit": 536870912, + "current": 437460992, + "peak": 490369024, + "anon": 410857472, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 106287104, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5328896, + "peak": 8384512, + "anon": 4112384, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 6876 + }, + { + "t": 363.8, + "containers": { + "docmost": { + "limit": 536870912, + "current": 436625408, + "peak": 490369024, + "anon": 410304512, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 106135552, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5332992, + "peak": 8384512, + "anon": 4112384, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 7176 + }, + { + "t": 379.0, + "containers": { + "docmost": { + "limit": 536870912, + "current": 436477952, + "peak": 490369024, + "anon": 410652672, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 106115072, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5341184, + "peak": 8384512, + "anon": 4112384, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 7472 + }, + { + "t": 394.1, + "containers": { + "docmost": { + "limit": 536870912, + "current": 437256192, + "peak": 490369024, + "anon": 410685440, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 106110976, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5349376, + "peak": 8384512, + "anon": 4112384, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 7772 + }, + { + "t": 409.3, + "containers": { + "docmost": { + "limit": 536870912, + "current": 437313536, + "peak": 490369024, + "anon": 410230784, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 106110976, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5615616, + "peak": 8384512, + "anon": 4108288, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 8072 + }, + { + "t": 424.5, + "containers": { + "docmost": { + "limit": 536870912, + "current": 435957760, + "peak": 490369024, + "anon": 409661440, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 103469056, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5099520, + "peak": 8384512, + "anon": 4108288, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 8372 + }, + { + "t": 439.7, + "containers": { + "docmost": { + "limit": 536870912, + "current": 437071872, + "peak": 490369024, + "anon": 410243072, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 99401728, + "peak": 131739648, + "anon": 11714560, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5115904, + "peak": 8384512, + "anon": 4120576, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 8672 + }, + { + "t": 454.8, + "containers": { + "docmost": { + "limit": 536870912, + "current": 436129792, + "peak": 490369024, + "anon": 410324992, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 98541568, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5120000, + "peak": 8384512, + "anon": 4116480, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 8968 + }, + { + "t": 470.0, + "containers": { + "docmost": { + "limit": 536870912, + "current": 438173696, + "peak": 490369024, + "anon": 412053504, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 98246656, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5652480, + "peak": 8384512, + "anon": 4120576, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 9268 + }, + { + "t": 485.1, + "containers": { + "docmost": { + "limit": 536870912, + "current": 434798592, + "peak": 490369024, + "anon": 408788992, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 97759232, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5140480, + "peak": 8531968, + "anon": 4124672, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 9568 + }, + { + "t": 500.3, + "containers": { + "docmost": { + "limit": 536870912, + "current": 437313536, + "peak": 490369024, + "anon": 410468352, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 97685504, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 6447104, + "peak": 8531968, + "anon": 4128768, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 9868 + }, + { + "t": 515.5, + "containers": { + "docmost": { + "limit": 536870912, + "current": 435871744, + "peak": 490369024, + "anon": 409559040, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 97681408, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5419008, + "peak": 8531968, + "anon": 4132864, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 10166 + }, + { + "t": 530.6, + "containers": { + "docmost": { + "limit": 536870912, + "current": 437919744, + "peak": 490369024, + "anon": 411553792, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 97689600, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5931008, + "peak": 8531968, + "anon": 4136960, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 10464 + }, + { + "t": 545.8, + "containers": { + "docmost": { + "limit": 536870912, + "current": 434409472, + "peak": 490369024, + "anon": 408412160, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 97120256, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5439488, + "peak": 8531968, + "anon": 4141056, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 10764 + }, + { + "t": 561.0, + "containers": { + "docmost": { + "limit": 536870912, + "current": 434151424, + "peak": 490369024, + "anon": 407638016, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 97124352, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5881856, + "peak": 8531968, + "anon": 4141056, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 11064 + }, + { + "t": 576.1, + "containers": { + "docmost": { + "limit": 536870912, + "current": 434028544, + "peak": 490369024, + "anon": 407797760, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 97128448, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5484544, + "peak": 8601600, + "anon": 4153344, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 11360 + }, + { + "t": 591.3, + "containers": { + "docmost": { + "limit": 536870912, + "current": 434966528, + "peak": 490369024, + "anon": 408449024, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 97120256, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5705728, + "peak": 8601600, + "anon": 4145152, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 11660 + }, + { + "t": 606.4, + "containers": { + "docmost": { + "limit": 536870912, + "current": 435929088, + "peak": 490369024, + "anon": 409374720, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-postgres": { + "limit": 268435456, + "current": 93388800, + "peak": 131739648, + "anon": 11677696, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + }, + "docmost-redis": { + "limit": 134217728, + "current": 5742592, + "peak": 8601600, + "anon": 4149248, + "oom_kill": 0, + "restarts": 0, + "oomkilled_flag": false, + "status": "running", + "cgroup": true + } + }, + "requests": 11960 + } +] \ No newline at end of file diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/migration-lines.txt b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/migration-lines.txt new file mode 100644 index 00000000..9a3d711f --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/migration-lines.txt @@ -0,0 +1 @@ +docmost | {"level":"info","time":"2026-09-30T20:26:24.623Z","pid":45,"hostname":"15f6b2bd7ca1","context":"DatabaseMigrationService","msg":"No pending database migrations"} \ No newline at end of file diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/run.log b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/run.log new file mode 100644 index 00000000..36b43ee8 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/run.log @@ -0,0 +1,58 @@ +[20:24:34] scratch drive folders cleared before FROM (R-656): none existed +[20:24:34] RT-docmost: deploying docmost at FROM {'docmost': 'docmost/docmost:0.96.0', 'docmost-postgres': 'postgres:18-alpine', 'docmost-redis': 'redis:7-alpine@sha256:c35af3bbcef51a62c8bae5a9a563c6f1b60d7ebaea4cb5a3ccbcc157580ae098'} +[20:25:58] FROM settled=True in 31.1s :: {"docmost": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "docmost-postgres": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "docmost-redis": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}} +[20:25:58] docmost: /api/auth/setup http=200 rc=0 +[20:25:58] docmost: login as the seeded user http=200 ok=True +[20:25:58] C1 (seed reads back BEFORE): True +[20:25:59] RT-docmost: swapping to TO {'docmost': 'docmost/docmost:0.96.0', 'docmost-postgres': 'postgres:18-alpine', 'docmost-redis': 'redis:7-alpine@sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499'} +[20:26:17] TO up -d rc=0 +[20:26:48] TO settled=True in 31.1s :: {"docmost": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "docmost-postgres": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}, "docmost-redis": {"status": "running", "health": "healthy", "restarts": 0, "exit": 0}} +[20:26:49] engine state docmost-postgres: 18 +[20:26:49] migration lines observed: 1 +[20:26:49] docmost: login as the seeded user http=200 ok=True +[20:26:49] RESULT (seed reads back AFTER): True +[20:26:49] memory watch: 600s, 4 callers on 1 path(s) at 172.18.0.3:3000 +[20:27:05] + 15s docmost=426M/512M peak=467M kills=0 rs=0 docmost-postgres=111M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=300 +[20:27:20] + 30s docmost=432M/512M peak=467M kills=0 rs=0 docmost-postgres=112M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=600 +[20:27:35] + 46s docmost=438M/512M peak=467M kills=0 rs=0 docmost-postgres=105M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=896 +[20:27:50] + 61s docmost=435M/512M peak=467M kills=0 rs=0 docmost-postgres=90M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=1196 +[20:28:05] + 76s docmost=423M/512M peak=467M kills=0 rs=0 docmost-postgres=76M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=1496 +[20:28:20] + 91s docmost=423M/512M peak=467M kills=0 rs=0 docmost-postgres=72M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=1796 +[20:28:36] + 106s docmost=420M/512M peak=467M kills=0 rs=0 docmost-postgres=72M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=2096 +[20:28:51] + 121s docmost=422M/512M peak=467M kills=0 rs=0 docmost-postgres=72M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=2392 +[20:29:06] + 136s docmost=423M/512M peak=467M kills=0 rs=0 docmost-postgres=72M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=2692 +[20:29:21] + 152s docmost=423M/512M peak=467M kills=0 rs=0 docmost-postgres=72M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=2992 +[20:29:36] + 167s docmost=422M/512M peak=467M kills=0 rs=0 docmost-postgres=72M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=3288 +[20:29:51] + 182s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=72M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=3588 +[20:30:07] + 197s docmost=420M/512M peak=467M kills=0 rs=0 docmost-postgres=72M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=3888 +[20:30:22] + 212s docmost=419M/512M peak=467M kills=0 rs=0 docmost-postgres=73M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=4184 +[20:30:37] + 227s docmost=418M/512M peak=467M kills=0 rs=0 docmost-postgres=73M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=4484 +[20:30:52] + 243s docmost=418M/512M peak=467M kills=0 rs=0 docmost-postgres=73M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=4784 +[20:31:07] + 258s docmost=416M/512M peak=467M kills=0 rs=0 docmost-postgres=107M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=5084 +[20:31:22] + 273s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=106M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=5380 +[20:31:38] + 288s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=106M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=5680 +[20:31:53] + 303s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=106M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=5980 +[20:32:08] + 318s docmost=416M/512M peak=467M kills=0 rs=0 docmost-postgres=106M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=6276 +[20:32:23] + 334s docmost=416M/512M peak=467M kills=0 rs=0 docmost-postgres=101M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=6576 +[20:32:38] + 349s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=101M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=6876 +[20:32:53] + 364s docmost=416M/512M peak=467M kills=0 rs=0 docmost-postgres=101M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=7176 +[20:33:08] + 379s docmost=416M/512M peak=467M kills=0 rs=0 docmost-postgres=101M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=7472 +[20:33:24] + 394s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=101M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=7772 +[20:33:39] + 409s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=101M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=8072 +[20:33:54] + 424s docmost=415M/512M peak=467M kills=0 rs=0 docmost-postgres=98M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=8372 +[20:34:09] + 440s docmost=416M/512M peak=467M kills=0 rs=0 docmost-postgres=94M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=8672 +[20:34:24] + 455s docmost=415M/512M peak=467M kills=0 rs=0 docmost-postgres=93M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=7M kills=0 rs=0 reqs=8968 +[20:34:39] + 470s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=93M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=7M kills=0 rs=0 reqs=9268 +[20:34:55] + 485s docmost=414M/512M peak=467M kills=0 rs=0 docmost-postgres=93M/256M peak=125M kills=0 rs=0 docmost-redis=4M/128M peak=8M kills=0 rs=0 reqs=9568 +[20:35:10] + 500s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=93M/256M peak=125M kills=0 rs=0 docmost-redis=6M/128M peak=8M kills=0 rs=0 reqs=9868 +[20:35:25] + 516s docmost=415M/512M peak=467M kills=0 rs=0 docmost-postgres=93M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=8M kills=0 rs=0 reqs=10166 +[20:35:40] + 531s docmost=417M/512M peak=467M kills=0 rs=0 docmost-postgres=93M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=8M kills=0 rs=0 reqs=10464 +[20:35:55] + 546s docmost=414M/512M peak=467M kills=0 rs=0 docmost-postgres=92M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=8M kills=0 rs=0 reqs=10764 +[20:36:10] + 561s docmost=414M/512M peak=467M kills=0 rs=0 docmost-postgres=92M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=8M kills=0 rs=0 reqs=11064 +[20:36:26] + 576s docmost=413M/512M peak=467M kills=0 rs=0 docmost-postgres=92M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=8M kills=0 rs=0 reqs=11360 +[20:36:41] + 591s docmost=414M/512M peak=467M kills=0 rs=0 docmost-postgres=92M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=8M kills=0 rs=0 reqs=11660 +[20:36:56] + 606s docmost=415M/512M peak=467M kills=0 rs=0 docmost-postgres=89M/256M peak=125M kills=0 rs=0 docmost-redis=5M/128M peak=8M kills=0 rs=0 reqs=11960 +[20:36:56] memory watch: killed=False tight=['docmost'] requests=11960 codes={'200': 11960} +[20:36:56] RT-docmost: ABORT — putting the FROM images back +[20:37:40] docmost: login as the seeded user http=200 ok=True +[20:37:40] ABORT: app came back in 31.1s; data present=True \ No newline at end of file diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/to-full.log b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/to-full.log new file mode 100644 index 00000000..ca4a4de6 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/to-full.log @@ -0,0 +1,86 @@ +docmost | $ pnpm --filter ./apps/server run start:prod +docmost | $ cross-env NODE_ENV=production node dist/main +docmost | (node:45) ExperimentalWarning: localStorage is not available because --localstorage-file was not provided. +docmost | (Use `node --trace-warnings ...` to show where the warning was created) +docmost | {"level":"info","time":"2026-09-30T20:26:24.177Z","pid":45,"hostname":"15f6b2bd7ca1","context":"RedisModule","msg":"default: the connection was successfully established"} +docmost | {"level":"info","time":"2026-09-30T20:26:24.430Z","pid":45,"hostname":"15f6b2bd7ca1","context":"DatabaseModule","msg":"Establishing database connection"} +docmost-redis | 1:C 30 Sep 2026 22:26:06.939 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. +docmost-redis | 1:C 30 Sep 2026 22:26:06.939 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo +docmost-redis | 1:C 30 Sep 2026 22:26:06.939 * Redis version=7.4.11, bits=64, commit=00000000, modified=0, pid=1, just started +docmost-redis | 1:C 30 Sep 2026 22:26:06.939 * Configuration loaded +docmost-postgres | The files belonging to this database system will be owned by user "postgres". +docmost-postgres | This user must also own the server process. +docmost-postgres | +docmost-postgres | The database cluster will be initialized with locale "en_US.utf8". +docmost-postgres | The default database encoding has accordingly been set to "UTF8". +docmost-postgres | The default text search configuration will be set to "english". +docmost-postgres | +docmost | {"level":"info","time":"2026-09-30T20:26:24.458Z","pid":45,"hostname":"15f6b2bd7ca1","context":"DatabaseModule","msg":"Database connection successful"} +docmost-postgres | Data page checksums are enabled. +docmost | {"level":"info","time":"2026-09-30T20:26:24.623Z","pid":45,"hostname":"15f6b2bd7ca1","context":"DatabaseMigrationService","msg":"No pending database migrations"} +docmost | {"level":"info","time":"2026-09-30T20:26:24.663Z","pid":45,"hostname":"15f6b2bd7ca1","context":"NestApplication","msg":"Nest application successfully started"} +docmost | {"level":"info","time":"2026-09-30T20:26:24.679Z","pid":45,"hostname":"15f6b2bd7ca1","context":"NestApplication","msg":"Listening on http://127.0.0.1:3000 / https://docs.gate.invalid"} +docmost-postgres | +docmost-postgres | fixing permissions on existing directory /var/lib/postgresql/18/docker ... ok +docmost-postgres | creating subdirectories ... ok +docmost-redis | 1:M 30 Sep 2026 22:26:06.939 * monotonic clock: POSIX clock_gettime +docmost-redis | 1:M 30 Sep 2026 22:26:06.941 * Running mode=standalone, port=6379. +docmost-redis | 1:M 30 Sep 2026 22:26:06.941 * Server initialized +docmost-redis | 1:M 30 Sep 2026 22:26:06.942 * Reading RDB base file on AOF loading... +docmost-redis | 1:M 30 Sep 2026 22:26:06.942 * Loading RDB produced by version 7.4.0 +docmost-redis | 1:M 30 Sep 2026 22:26:06.942 * RDB age 50 seconds +docmost-redis | 1:M 30 Sep 2026 22:26:06.942 * RDB memory usage when created 0.90 Mb +docmost-redis | 1:M 30 Sep 2026 22:26:06.942 * RDB is base AOF +docmost-redis | 1:M 30 Sep 2026 22:26:06.942 * Done loading RDB, keys loaded: 0, keys expired: 0. +docmost-redis | 1:M 30 Sep 2026 22:26:06.942 * DB loaded from base file appendonly.aof.1.base.rdb: 0.001 seconds +docmost-postgres | selecting dynamic shared memory implementation ... posix +docmost-postgres | selecting default "max_connections" ... 100 +docmost-redis | 1:M 30 Sep 2026 22:26:06.943 * DB loaded from incr file appendonly.aof.1.incr.aof: 0.001 seconds +docmost-redis | 1:M 30 Sep 2026 22:26:06.943 * DB loaded from append only file: 0.002 seconds +docmost-redis | 1:M 30 Sep 2026 22:26:06.943 * Opening AOF incr file appendonly.aof.1.incr.aof on server start +docmost-redis | 1:M 30 Sep 2026 22:26:06.943 * Ready to accept connections tcp +docmost-postgres | selecting default "shared_buffers" ... 128MB +docmost-postgres | selecting default time zone ... Europe/Budapest +docmost-postgres | creating configuration files ... ok +docmost-postgres | running bootstrap script ... ok +docmost-postgres | sh: locale: not found +docmost-postgres | 2026-09-30 22:25:16.617 CEST [40] WARNING: no usable system locales were found +docmost-postgres | performing post-bootstrap initialization ... ok +docmost-postgres | syncing data to disk ... ok +docmost-postgres | +docmost-postgres | +docmost-postgres | Success. You can now start the database server using: +docmost-postgres | +docmost-postgres | pg_ctl -D /var/lib/postgresql/18/docker -l logfile start +docmost-postgres | +docmost-postgres | initdb: warning: enabling "trust" authentication for local connections +docmost-postgres | initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. +docmost-postgres | waiting for server to start....2026-09-30 22:25:17.456 CEST [46] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-musl, compiled by gcc (Alpine 15.2.0) 15.2.0, 64-bit +docmost-postgres | 2026-09-30 22:25:17.460 CEST [46] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" +docmost-postgres | 2026-09-30 22:25:17.479 CEST [52] LOG: database system was shut down at 2026-09-30 22:25:17 CEST +docmost-postgres | 2026-09-30 22:25:17.488 CEST [46] LOG: database system is ready to accept connections +docmost-postgres | done +docmost-postgres | server started +docmost-postgres | CREATE DATABASE +docmost-postgres | +docmost-postgres | +docmost-postgres | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/* +docmost-postgres | +docmost-postgres | waiting for server to shut down....2026-09-30 22:25:17.648 CEST [46] LOG: received fast shutdown request +docmost-postgres | 2026-09-30 22:25:17.654 CEST [46] LOG: aborting any active transactions +docmost-postgres | 2026-09-30 22:25:17.657 CEST [46] LOG: background worker "logical replication launcher" (PID 55) exited with exit code 1 +docmost-postgres | 2026-09-30 22:25:17.659 CEST [50] LOG: shutting down +docmost-postgres | 2026-09-30 22:25:17.664 CEST [50] LOG: checkpoint starting: shutdown immediate +docmost-postgres | 2026-09-30 22:25:17.752 CEST [50] LOG: checkpoint complete: wrote 943 buffers (5.8%), wrote 3 SLRU buffers; 0 WAL file(s) added, 0 removed, 0 recycled; write=0.034 s, sync=0.033 s, total=0.093 s; sync files=303, longest=0.004 s, average=0.001 s; distance=4362 kB, estimate=4362 kB; lsn=0/1BA8858, redo lsn=0/1BA8858 +docmost-postgres | 2026-09-30 22:25:17.776 CEST [46] LOG: database system is shut down +docmost-postgres | done +docmost-postgres | server stopped +docmost-postgres | +docmost-postgres | PostgreSQL init process complete; ready for start up. +docmost-postgres | +docmost-postgres | 2026-09-30 22:25:17.897 CEST [1] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-musl, compiled by gcc (Alpine 15.2.0) 15.2.0, 64-bit +docmost-postgres | 2026-09-30 22:25:17.897 CEST [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 +docmost-postgres | 2026-09-30 22:25:17.897 CEST [1] LOG: listening on IPv6 address "::", port 5432 +docmost-postgres | 2026-09-30 22:25:17.905 CEST [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" +docmost-postgres | 2026-09-30 22:25:17.918 CEST [68] LOG: database system was shut down at 2026-09-30 22:25:17 CEST +docmost-postgres | 2026-09-30 22:25:17.929 CEST [1] LOG: database system is ready to accept connections diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/to-states.json b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/to-states.json new file mode 100644 index 00000000..c6bae4a5 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/to-states.json @@ -0,0 +1,20 @@ +{ + "docmost": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + }, + "docmost-postgres": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + }, + "docmost-redis": { + "status": "running", + "health": "healthy", + "restarts": 0, + "exit": 0 + } +} \ No newline at end of file diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/verdict.json b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/verdict.json new file mode 100644 index 00000000..4a9155b2 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/bench/evidence/RT-docmost/verdict.json @@ -0,0 +1,87 @@ +{ + "harness_version": 4, + "edge": "RT-docmost", + "app": "docmost", + "note": "re-test of the same tag at a new digest (decision 52): docmost-redis sha256:c35af3bbcef5 -> sha256:858f009f9709", + "from": { + "docmost": "docmost/docmost:0.96.0", + "docmost-postgres": "postgres:18-alpine", + "docmost-redis": "redis:7-alpine@sha256:c35af3bbcef51a62c8bae5a9a563c6f1b60d7ebaea4cb5a3ccbcc157580ae098" + }, + "to": { + "docmost": "docmost/docmost:0.96.0", + "docmost-postgres": "postgres:18-alpine", + "docmost-redis": "redis:7-alpine@sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499" + }, + "verdict": "proven", + "seed_read_before": true, + "seed_read_after": true, + "healthy_after": true, + "migration_observed": "\u001b[2Kdocmost | {\"level\":\"info\",\"time\":\"2026-09-30T20:26:24.623Z\",\"pid\":45,\"hostname\":\"15f6b2bd7ca1\",\"context\":\"DatabaseMigrationService\",\"msg\":\"No pending database migrations\"}", + "abort": "starts-and-serves", + "abort_detail": null, + "engine_state_after": { + "docmost-postgres": { + "image": "postgres:18-alpine", + "probe": "datadir major version", + "answer": "18", + "probe_rc": 0 + } + }, + "memory": { + "soak_s": 606.7, + "requested_s": 600, + "requests": 11960, + "codes": { + "200": 11960 + }, + "first_kill": null, + "containers": { + "docmost": { + "limit": 536870912, + "peak": 490369024, + "peak_pct": 0.913, + "anon_peak_sampled": 432144384, + "anon_peak_pct": 0.805, + "oom_kills": 0, + "restarts": 0, + "oomkilled_flag": false, + "measured": true + }, + "docmost-postgres": { + "limit": 268435456, + "peak": 131739648, + "peak_pct": 0.491, + "anon_peak_sampled": 11714560, + "anon_peak_pct": 0.044, + "oom_kills": 0, + "restarts": 0, + "oomkilled_flag": false, + "measured": true + }, + "docmost-redis": { + "limit": 134217728, + "peak": 8601600, + "peak_pct": 0.064, + "anon_peak_sampled": 4161536, + "anon_peak_pct": 0.031, + "oom_kills": 0, + "restarts": 0, + "oomkilled_flag": false, + "measured": true + } + }, + "unmeasured": [], + "load": "reached" + }, + "marks": [ + "memory_tight" + ], + "duration_s": 31.1, + "measured_at": "2026-09-30T20:37:40Z", + "evidence": "evidence/RT-docmost", + "scratch_cleared": [], + "files_changed": [], + "files_changed_detail": [], + "total_s": 785.1 +} \ No newline at end of file diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/box-outline-attempt-fixture-fault/box-verdict-outline.json b/documentation/audits/night-rulings-2026-09-30/A/e2e/box-outline-attempt-fixture-fault/box-verdict-outline.json new file mode 100644 index 00000000..33d38f88 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/box-outline-attempt-fixture-fault/box-verdict-outline.json @@ -0,0 +1,13 @@ +{ + "app": "outline", + "verdict": "failed", + "venue": "box 9202 (drill catalog), chain", + "retested": { + "outline-redis": { + "ref": "redis:7-alpine", + "from_digest": "sha256:c35af3bbcef51a62c8bae5a9a563c6f1b60d7ebaea4cb5a3ccbcc157580ae098", + "to_digest": "sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499" + } + }, + "why": "C1: the fixture could not seed and read back before the re-test" +} \ No newline at end of file diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/box-outline-attempt-fixture-fault/box.txt b/documentation/audits/night-rulings-2026-09-30/A/e2e/box-outline-attempt-fixture-fault/box.txt new file mode 100644 index 00000000..1cac49c1 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/box-outline-attempt-fixture-fault/box.txt @@ -0,0 +1,3 @@ +running digests after install: {'outline-redis': 'sha256:c35af3bbcef51a62c8bae5a9a563c6f1b60d7ebaea4cb5a3ccbcc157580ae098'} + outline: installation.create http=302 +RESULT failed — C1: the fixture could not seed and read back before the re-test diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/box/box-verdict-docmost.json b/documentation/audits/night-rulings-2026-09-30/A/e2e/box/box-verdict-docmost.json new file mode 100644 index 00000000..509fb4d9 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/box/box-verdict-docmost.json @@ -0,0 +1,71 @@ +{ + "app": "docmost", + "verdict": "proven", + "venue": "box 9202 (drill catalog), chain", + "retested": { + "docmost-redis": { + "ref": "redis:7-alpine", + "from_digest": "sha256:c35af3bbcef51a62c8bae5a9a563c6f1b60d7ebaea4cb5a3ccbcc157580ae098", + "to_digest": "sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499" + } + }, + "seed_read_before": true, + "badge_before": { + "hu": [ + { + "title": "\u00dajabb v\u00e1ltozat \u00e9rhet\u0151 el ehhez az alkalmaz\u00e1shoz. A friss\u00edt\u00e9s ind\u00edt\u00e1s\u00e1hoz nyomd meg a Friss\u00edt\u00e9s gombot.", + "text": "Friss\u00edt\u00e9s el\u00e9rhet\u0151 \u2014 ma" + } + ], + "en": [ + { + "title": "A newer version of this app is available. Select the Update button to start it.", + "text": "Update available \u2014 today" + } + ] + }, + "leg_log": [ + "2026/09/30 20:39:43 night_chain.go:108: [INFO] [night-chain] manual run started from 172.18.0.6:40920: [db-dump tier2 update-leg]", + "2026/09/30 20:40:25 night_chain.go:77: [INFO] [night-chain] update-leg: started", + "2026/09/30 20:40:25 unattended.go:270: [INFO] [update-leg] started (manual-chain): window 02:30, no step starts at or after 01:55", + "2026/09/30 20:40:25 unattended.go:336: [INFO] [update-leg] docmost: step pressed docmost=docmost/docmost:0.96.0, docmost-postgres=postgres:18-alpine, docmost-redis=redis:7-alpine \u2192 docmost=docmost/docmost:0.96.0, docmost-postgres=postgres:18-alpine, docmost-redis=redis:7-alpine", + "2026/09/30 20:42:00 unattended.go:343: [INFO] [update-leg] docmost: step ended done after 95.0 s", + "2026/09/30 20:42:00 unattended.go:316: [INFO] [update-leg] privatebin: skipped \u2014 stopped_by_household", + "2026/09/30 20:42:00 unattended.go:248: [INFO] [update-leg] update leg (manual-chain): done=1 undone=0 held=0 failed=0 skipped=1 in 1m35s [skipped: privatebin=stopped_by_household]", + "2026/09/30 20:42:00 night_chain.go:82: [INFO] [night-chain] update-leg: done in 1m35s" + ], + "final_phase": "done", + "digests_before": { + "docmost-redis": "sha256:c35af3bbcef51a62c8bae5a9a563c6f1b60d7ebaea4cb5a3ccbcc157580ae098" + }, + "digests_after": { + "docmost-redis": "sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499" + }, + "seed_read_after": true, + "badge_after": { + "hu": [ + { + "title": "Ez az alkalmaz\u00e1s a legfrissebb el\u00e9rhet\u0151 v\u00e1ltozatot futtatja.", + "text": "Naprak\u00e9sz" + } + ], + "en": [ + { + "title": "This app is running the newest version available.", + "text": "Up to date" + } + ] + }, + "from": { + "docmost": "docmost/docmost:0.96.0", + "docmost-postgres": "postgres:18-alpine", + "docmost-redis": "redis:7-alpine" + }, + "to": { + "docmost": "docmost/docmost:0.96.0", + "docmost-postgres": "postgres:18-alpine", + "docmost-redis": "redis:7-alpine" + }, + "measured_at": "2026-09-30T20:39:42Z", + "why": "the re-test step ran on the box" +} \ No newline at end of file diff --git a/documentation/audits/night-rulings-2026-09-30/A/e2e/box/box.txt b/documentation/audits/night-rulings-2026-09-30/A/e2e/box/box.txt new file mode 100644 index 00000000..f001d404 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/A/e2e/box/box.txt @@ -0,0 +1,20 @@ +running digests after install: {'docmost-redis': 'sha256:c35af3bbcef51a62c8bae5a9a563c6f1b60d7ebaea4cb5a3ccbcc157580ae098'} + docmost: /api/auth/setup http=200 rc=0 + docmost: login as the seeded user http=200 ok=True +drill: 429563b DRILL docmost: re-test of the same tag {'docmost-redis': 'sha256:858f009f9709'} (box proof) +the box's catalog_digests carry the new digest after 1 sync round(s) +badge before: {'hu': [{'title': 'Újabb változat érhető el ehhez az alkalmazáshoz. A frissítés indításához nyomd meg a Frissítés gombot.', 'text': 'Frissítés elérhető — ma'}], 'en': [{'title': 'A newer version of this app is available. Select the Update button to start it.', 'text': 'Update available — today'}]} +night chain -> 202 {'data': {'legs': ['db-dump', 'tier2', 'update-leg']}, 'message': 'started', 'ok': True} +the leg's own lines: +2026/09/30 20:39:43 night_chain.go:108: [INFO] [night-chain] manual run started from 172.18.0.6:40920: [db-dump tier2 update-leg] +2026/09/30 20:40:25 night_chain.go:77: [INFO] [night-chain] update-leg: started +2026/09/30 20:40:25 unattended.go:270: [INFO] [update-leg] started (manual-chain): window 02:30, no step starts at or after 01:55 +2026/09/30 20:40:25 unattended.go:336: [INFO] [update-leg] docmost: step pressed docmost=docmost/docmost:0.96.0, docmost-postgres=postgres:18-alpine, docmost-redis=redis:7-alpine → docmost=docmost/docmost:0.96.0, docmost-postgres=postgres:18-alpine, docmost-redis=redis:7-alpine +2026/09/30 20:42:00 unattended.go:343: [INFO] [update-leg] docmost: step ended done after 95.0 s +2026/09/30 20:42:00 unattended.go:316: [INFO] [update-leg] privatebin: skipped — stopped_by_household +2026/09/30 20:42:00 unattended.go:248: [INFO] [update-leg] update leg (manual-chain): done=1 undone=0 held=0 failed=0 skipped=1 in 1m35s [skipped: privatebin=stopped_by_household] +2026/09/30 20:42:00 night_chain.go:82: [INFO] [night-chain] update-leg: done in 1m35s + + docmost: login as the seeded user http=200 ok=True +after: phase=done digests={'docmost-redis': 'sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499'} read_back=True badge={'hu': [{'title': 'Ez az alkalmazás a legfrissebb elérhető változatot futtatja.', 'text': 'Naprakész'}], 'en': [{'title': 'This app is running the newest version available.', 'text': 'Up to date'}]} +RESULT proven — the re-test step ran on the box diff --git a/documentation/audits/night-rulings-2026-09-30/B/B1-red-proofs.txt b/documentation/audits/night-rulings-2026-09-30/B/B1-red-proofs.txt new file mode 100644 index 00000000..bec3cec3 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/B/B1-red-proofs.txt @@ -0,0 +1,16 @@ +TestImageRetention_KeepsRunningAndPreviousDeletesOlder: ran=True RED ["image_retention_test.go:113: the undo's image (web:2) was deleted"] +TestImageRetention_AStoppedAppsComposeKeepsItsImage: ran=True RED ["image_retention_test.go:176: a stopped app's image was deleted although its compose names it"] +TestImageRetention_NoPassWhileAnUpdateRuns: ran=True RED ['image_retention_test.go:242: a pass ran while docs was updating: [sha256:P16 sha256:W1]'] +TestImageRetention_UnreadableKeepSetDeletesNothing: ran=True RED ['image_retention_test.go:194: no error from an unreadable keep set'] +restored: ok gitea.dooplex.hu/admin/felhom-controller/internal/stacks 0.154s +=== RUN TestImageRetention_ASharedImageSurvivesTheRemoveOfOneApp + image_retention_test.go:147: the shared postgres:18-alpine was deleted while docs still runs it +FAIL +TestImageRetention_TheRemoveButtonRunsIt: ran=False RED [] +TestImageRetention_ADoneUpdateRunsItWithThePrevious: ran=True RED ['image_retention_test.go:290: the done update did not run the retention'] +=== RUN TestImageRetention_TheRemoveButtonRunsIt + image_retention_test.go:261: RemoveStack did not run the retention with the app's repos (got "", map[]) +FAIL +=== RUN TestImageRetention_SeesUntaggedDigestPulledImages + image_retention_test.go:315: an untagged old image was not deleted: sha256:P16,sha256:W1 +FAIL diff --git a/documentation/audits/night-rulings-2026-09-30/B/B2-9202-images-before.txt b/documentation/audits/night-rulings-2026-09-30/B/B2-9202-images-before.txt new file mode 100644 index 00000000..b3c92322 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/B/B2-9202-images-before.txt @@ -0,0 +1,22 @@ +Wed Sep 30 20:35:46 UTC 2026 +TYPE TOTAL ACTIVE SIZE RECLAIMABLE +Images 39 6 23.79GB 20.69GB (86%) +Containers 6 6 28.77MB 0B (0%) +Local Volumes 249 5 103.1MB 15.98kB (0%) +Build Cache 4 0 262B 262B + +alpine:3.20 7.81MB +alpine:latest 8.42MB +curlimages/curl:8.11.1 21.8MB +flomp/wanderer-db:v0.20.0 45.3MB +flomp/wanderer-web:v0.20.0 708MB +getmeili/meilisearch:v1.36.0 158MB +ghcr.io/paperless-ngx/paperless-ngx:2.20.15 1.42GB +gitea.dooplex.hu/admin/felhom-controller:0.283.1 409MB +gtstef/filebrowser:1.3.3-stable 215MB +postgres:18-alpine 304MB +redis:7-alpine 39.1MB +redis:7.4-alpine 39.1MB +traefik:v3.6.7 186MB + +/dev/loop1 69G 26G 39G 41% /var/lib/docker diff --git a/documentation/audits/night-rulings-2026-09-30/B/B3-9202-deploy.txt b/documentation/audits/night-rulings-2026-09-30/B/B3-9202-deploy.txt new file mode 100644 index 00000000..82c30d67 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/B/B3-9202-deploy.txt @@ -0,0 +1,2 @@ +gitea.dooplex.hu/admin/felhom-controller:0.284.0 +gitea.dooplex.hu/admin/felhom-controller:0.284.0 Up 20 seconds (healthy) diff --git a/documentation/audits/night-rulings-2026-09-30/B/B4-9202-retention-after.txt b/documentation/audits/night-rulings-2026-09-30/B/B4-9202-retention-after.txt new file mode 100644 index 00000000..909eb7ee --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/B/B4-9202-retention-after.txt @@ -0,0 +1,11 @@ +2026/09/30 20:38:50 image_retention.go:238: [INFO] [stacks] image retention (one-time clean-up): deleted [flomp/wanderer-db:v0.20.0] (645f630a4e81, 45.3MB) — no container, installed app or undo names it (decision 53) +2026/09/30 20:38:51 image_retention.go:238: [INFO] [stacks] image retention (one-time clean-up): deleted [flomp/wanderer-web:v0.20.0] (8b4ef575b264, 708MB) — no container, installed app or undo names it (decision 53) +2026/09/30 20:38:52 image_retention.go:238: [INFO] [stacks] image retention (one-time clean-up): deleted [getmeili/meilisearch:v1.36.0] (9d728eb1a254, 158MB) — no container, installed app or undo names it (decision 53) +2026/09/30 20:38:52 image_retention.go:348: [INFO] [stacks] image retention (one-time): deleted 3 image(s). docker disk before: Images 25.55GB 20.72GB (81%) | after: Images 24.64GB 19.82GB (80%) + +Wed Sep 30 20:44:43 UTC 2026 +TYPE TOTAL ACTIVE SIZE RECLAIMABLE +Images 40 8 26.56GB 20.99GB (79%) +redis:7-alpine f84b0c467801 39.1MB +redis:7.4-alpine f84b0c467801 39.1MB +/dev/loop1 69G 30G 36G 45% /var/lib/docker diff --git a/documentation/audits/night-rulings-2026-09-30/B/B5-9202-0.284.1-before-removes.txt b/documentation/audits/night-rulings-2026-09-30/B/B5-9202-0.284.1-before-removes.txt new file mode 100644 index 00000000..ed31a402 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/B/B5-9202-0.284.1-before-removes.txt @@ -0,0 +1,5 @@ +gitea.dooplex.hu/admin/felhom-controller:0.284.1 Up 25 seconds (healthy) + +redis:7-alpine sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499 39.1MB +redis:7.4-alpine sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499 39.1MB +postgres:18-alpine sha256:77f585114c32fbca283dc835b0596f4e52b51b4c6662d7810b2f4084f60a1873 304MB diff --git a/documentation/audits/night-rulings-2026-09-30/B/B6-9202-0.284.2-deploy.txt b/documentation/audits/night-rulings-2026-09-30/B/B6-9202-0.284.2-deploy.txt new file mode 100644 index 00000000..3fa9dd5e --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/B/B6-9202-0.284.2-deploy.txt @@ -0,0 +1,3 @@ +Images 41 8 26.6GB 21.02GB (79%) +41 +gitea.dooplex.hu/admin/felhom-controller:0.284.2 Up 25 seconds (healthy) diff --git a/documentation/audits/night-rulings-2026-09-30/B/B7-9202-one-time-sweep-v2.txt b/documentation/audits/night-rulings-2026-09-30/B/B7-9202-one-time-sweep-v2.txt new file mode 100644 index 00000000..6e862e84 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/B/B7-9202-one-time-sweep-v2.txt @@ -0,0 +1,59 @@ +2026/09/30 21:07:31 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [lscr.io/linuxserver/radarr:] (17292112a802, 211MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:07:32 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [rommapp/romm:] (26b2c8009feb, 775MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:07:40 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [n8nio/n8n:] (2a8ec1d5dcf1, 1.04GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:07:40 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [deluan/navidrome:] (2b2799fc3308, 250MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:07:47 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [n8nio/n8n:] (386b575a2aa5, 1.04GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:07:48 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [lukevella/rallly:] (490d37d14930, 1.01GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:07:53 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [n8nio/n8n:] (4f6a979c4566, 1.04GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:01 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [ghcr.io/home-assistant/home-assistant:] (5157d3b45f23, 2.34GB) — no container, installed app or undo names it (decis +2026/09/30 21:08:01 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [redis:] (5509c0097c60, 39.1MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:02 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [kimai/kimai2:] (63c57887594e, 877MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:03 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [grafana/grafana:] (747aaf291145, 1.4GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:03 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [postgres:] (75f5a96988cd, 294MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:03 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [emby/embyserver:] (829e7095223b, 1GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:06 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [outlinewiki/outline:] (93c7d0025bdc, 1.03GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:06 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [ghcr.io/advplyr/audiobookshelf:] (a78da6dbcaa6, 320MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:09 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [outlinewiki/outline:] (ab2c5a0052eb, 1.09GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:12 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [ghcr.io/immich-app/immich-server:] (b1a1cc8cac12, 1.75GB) — no container, installed app or undo names it (decision 5 +2026/09/30 21:08:12 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [lscr.io/linuxserver/sonarr:] (be3661de689b, 203MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:12 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [gotson/komga:] (bed1c0312c9a, 483MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:14 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [ghcr.io/immich-app/immich-machine-learning:] (c53e2e5c4e94, 952MB) — no container, installed app or undo names it (d +2026/09/30 21:08:15 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [actualbudget/actual-server:] (cfacf9b19b21, 359MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:17 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [ghcr.io/mealie-recipes/mealie:] (dc61bc0635ee, 1.17GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:19 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [ghcr.io/mealie-recipes/mealie:] (e966ee76a4c1, 1.2GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:20 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [rommapp/romm:] (eab97d88d9d7, 776MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:25 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [ghost:] (f054a31295f3, 634MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:26 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [lscr.io/linuxserver/bookstack:] (f70e954bca33, 415MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:26 image_retention.go:222: [INFO] [stacks] image retention (one-time clean-up): pass over 42 image(s) of [actualbudget/actual-server calcom/cal.com codewithcj/sparkyfitness codewithcj/sparkyfitness_server crocodil +2026/09/30 21:08:26 image_retention.go:364: [INFO] [stacks] image retention (one-time): deleted 26 image(s). docker disk before: Images 26.63GB 21.06GB (79%) | after: Images 5.684GB 637.6MB (11%) +Images 16 8 5.684GB 637.6MB (11%) +16 +/dev/loop1 69G 6.3G 59G 10% /var/lib/docker +felhom-controller Up 4 minutes (healthy) +filebrowser Up 13 minutes (healthy) +docmost-redis Up 28 minutes (healthy) +paperless-webserver Up 28 minutes (healthy) +paperless-postgres Up 28 minutes (healthy) +paperless-redis Up 28 minutes (healthy) +docmost Up 28 minutes (healthy) +docmost-postgres Up 28 minutes (healthy) +calibre-web Up 28 minutes (healthy) +traefik Up 6 days + +c187fea3a3b2 gitea.dooplex.hu/admin/felhom-controller:0.284.2 sha256:11da4e419e0876ad3288375d6105ed774d5f2317f +4d046ec22473 gitea.dooplex.hu/admin/felhom-controller:0.284.1 sha256:60e8135d93af3a6d9195621c6abb4bf17aa72e5b7 +67fbed846822 gitea.dooplex.hu/admin/felhom-controller:0.284.0 sha256:73c251757f52c0cc2c4ef1f6b668a4af27c2dedb3 +79d28d57f414 gitea.dooplex.hu/admin/felhom-controller:0.283.1 sha256:5cb27bd39a7c0c23e2a5604813f0b2f6737c8d248 +43505abb1b0e crocodilestick/calibre-web-automated: sha256:5e00373854247750cc3e4479b492ae09293ff5e06ed101 +f84b0c467801 redis:7-alpine sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499 +f84b0c467801 redis:7.4-alpine sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499 +c293117fcecd postgres:18-alpine sha256:77f585114c32fbca283dc835b0596f4e52b51b4c6662d7810b2f4084f60a1873 +320994c3b997 alpine:latest sha256:294b683cb724975bec92580e1e685676bd4b50bda910ddb8c51d4cabeaec77e6 +0f146d72f36f docmost/docmost: sha256:b56947fcfd08aab8fae12a377e1792784786adbf8b96e4281f14ef4fc072685a +1979d2941666 : +095fd20d87be gtstef/filebrowser:1.3.3-stable sha256:eb3733681db8757412632c61a99ad656f0d94ed6781bb2ea114b4d70ba +623d64064a8d ghcr.io/paperless-ngx/paperless-ngx:2.20.15 sha256:6c86cad803970ea782683a8e80e7403444c5bf3cf70de6 +bf8527eb54c3 alpine:3.20 sha256:d9e853e87e55526f6b2917df91a2115c36dd7c696a35be12163d44e6e2a4b6bc +91528df1690f traefik:v3.6.7 sha256:a9890c898f379c1905ee5b28342f6b408dc863f08db2dab20e46c267d1ff463a +7551dbeefe0d curlimages/curl:8.11.1 sha256:c1fe1679c34d9784c1b0d1e5f62ac0a79fca01fb6377cdd33e90473c6f9f9a69 +7d06252fad43 redis: sha256:c35af3bbcef51a62c8bae5a9a563c6f1b60d7ebaea4cb5a3ccbcc157580ae098 diff --git a/documentation/audits/night-rulings-2026-09-30/B/B8-9202-removes.txt b/documentation/audits/night-rulings-2026-09-30/B/B8-9202-removes.txt new file mode 100644 index 00000000..211f7e51 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/B/B8-9202-removes.txt @@ -0,0 +1,57 @@ +23:08:57 == remove calibre-web +23:09:42 [X] after remove: deployed=False leftovers='/opt/docker/stacks/calibre-web' +23:09:42 == remove docmost +23:10:23 [X] after remove: deployed=False leftovers='/opt/docker/stacks/docmost' +2026/09/30 21:07:31 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [lscr.io/linuxserver/radarr:] (17292112a802, 211MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:07:32 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [rommapp/romm:] (26b2c8009feb, 775MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:07:40 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [n8nio/n8n:] (2a8ec1d5dcf1, 1.04GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:07:40 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [deluan/navidrome:] (2b2799fc3308, 250MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:07:47 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [n8nio/n8n:] (386b575a2aa5, 1.04GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:07:48 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [lukevella/rallly:] (490d37d14930, 1.01GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:07:53 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [n8nio/n8n:] (4f6a979c4566, 1.04GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:01 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [ghcr.io/home-assistant/home-assistant:] (5157d3b45f23, 2.34GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:01 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [redis:] (5509c0097c60, 39.1MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:02 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [kimai/kimai2:] (63c57887594e, 877MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:03 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [grafana/grafana:] (747aaf291145, 1.4GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:03 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [postgres:] (75f5a96988cd, 294MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:03 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [emby/embyserver:] (829e7095223b, 1GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:06 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [outlinewiki/outline:] (93c7d0025bdc, 1.03GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:06 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [ghcr.io/advplyr/audiobookshelf:] (a78da6dbcaa6, 320MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:09 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [outlinewiki/outline:] (ab2c5a0052eb, 1.09GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:12 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [ghcr.io/immich-app/immich-server:] (b1a1cc8cac12, 1.75GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:12 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [lscr.io/linuxserver/sonarr:] (be3661de689b, 203MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:12 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [gotson/komga:] (bed1c0312c9a, 483MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:14 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [ghcr.io/immich-app/immich-machine-learning:] (c53e2e5c4e94, 952MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:15 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [actualbudget/actual-server:] (cfacf9b19b21, 359MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:17 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [ghcr.io/mealie-recipes/mealie:] (dc61bc0635ee, 1.17GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:19 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [ghcr.io/mealie-recipes/mealie:] (e966ee76a4c1, 1.2GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:20 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [rommapp/romm:] (eab97d88d9d7, 776MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:25 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [ghost:] (f054a31295f3, 634MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:26 image_retention.go:249: [INFO] [stacks] image retention (one-time clean-up): deleted [lscr.io/linuxserver/bookstack:] (f70e954bca33, 415MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:08:26 image_retention.go:222: [INFO] [stacks] image retention (one-time clean-up): pass over 42 image(s) of [actualbudget/actual-server calcom/cal.com codewithcj/sparkyfitness codewithcj/sparkyfitness_server crocodilestick/calibre-web-a +2026/09/30 21:08:26 image_retention.go:364: [INFO] [stacks] image retention (one-time): deleted 26 image(s). docker disk before: Images 26.63GB 21.06GB (79%) | after: Images 5.684GB 637.6MB (11%) +2026/09/30 21:09:35 image_retention.go:249: [INFO] [stacks] image retention (remove of calibre-web): deleted [crocodilestick/calibre-web-automated:] (43505abb1b0e, 1.72GB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:09:35 image_retention.go:222: [INFO] [stacks] image retention (remove of calibre-web): pass over 16 image(s) of [crocodilestick/calibre-web-automated] — 1 candidate(s), 1 deleted, the rest kept +2026/09/30 21:10:18 image_retention.go:249: [INFO] [stacks] image retention (remove of docmost): deleted [docmost/docmost:] (0f146d72f36f, 753MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:10:18 image_retention.go:249: [INFO] [stacks] image retention (remove of docmost): deleted [redis:] (7d06252fad43, 41.2MB) — no container, installed app or undo names it (decision 53) +2026/09/30 21:10:18 image_retention.go:222: [INFO] [stacks] image retention (remove of docmost): pass over 15 image(s) of [docmost/docmost postgres redis] — 2 candidate(s), 2 deleted, the rest kept +: +alpine:3.20 +alpine:latest +curlimages/curl:8.11.1 +ghcr.io/paperless-ngx/paperless-ngx:2.20.15 +gitea.dooplex.hu/admin/felhom-controller:0.283.1 +gitea.dooplex.hu/admin/felhom-controller:0.284.0 +gitea.dooplex.hu/admin/felhom-controller:0.284.1 +gitea.dooplex.hu/admin/felhom-controller:0.284.2 +gtstef/filebrowser:1.3.3-stable +postgres:18-alpine +redis:7-alpine +redis:7.4-alpine +traefik:v3.6.7 +felhom-controller Up 5 minutes (healthy) +filebrowser Up 15 minutes (healthy) +paperless-webserver Up 30 minutes (healthy) +paperless-postgres Up 30 minutes (healthy) +paperless-redis Up 30 minutes (healthy) +traefik Up 6 days diff --git a/documentation/audits/night-rulings-2026-09-30/C/C1-red-proofs.txt b/documentation/audits/night-rulings-2026-09-30/C/C1-red-proofs.txt new file mode 100644 index 00000000..1facf0bc --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/C/C1-red-proofs.txt @@ -0,0 +1,10 @@ +TestInstallHold_WrittenBeforeTheFirstStart: sabotage 'deploy.go' -> ran=True rc=1 RED ['install_hold_test.go:49: the hold file did not exist when the app was first started — its known default login was reachable (R-741)'] +TestInstallHold_OpensWhenAfterInstallSucceeds: sabotage 'after_install.go' -> ran=True rc=1 RED ['install_hold_test.go:95: the hold file is still there after the login was replaced'] +TestInstallHold_FailureKeepsItTheHouseholdOpensIt: sabotage 'setup_gate.go' -> ran=True rc=1 RED ["install_hold_test.go:128: the household's word did not open the hold"] +TestInstallHold_ReRunsOnlyAnInstallTheRestartCutOff: sabotage 'install_hold.go' -> ran=True rc=1 RED ['install_hold_test.go:183: the loop re-ran after_install for an install this process made — twice at once'] +restored: FAIL +=== RUN TestInstallHold_ReRunsOnlyAnInstallTheRestartCutOff + install_hold_test.go:183: the loop re-ran after_install for an install this process made — twice at once +FAIL +ok gitea.dooplex.hu/admin/felhom-controller/internal/stacks 0.119s +(the first run's 'restored: FAIL' was the DeployedAt whole-second edge, fixed: compare with the process start truncated to the second) diff --git a/documentation/audits/night-rulings-2026-09-30/C/C2-calibre-web-poll.txt b/documentation/audits/night-rulings-2026-09-30/C/C2-calibre-web-poll.txt new file mode 100644 index 00000000..c506b66d --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/C/C2-calibre-web-poll.txt @@ -0,0 +1,21 @@ +# calibre-web: stranger polls of /opds with the DEFAULT login, from the deploy press (controller gitea.dooplex.hu/admin/felhom-controller:0.284.0) +deploy: True +20:36:29 default login -> 404 +20:37:16 default login -> 401 +20:37:18 default login -> 403 +20:37:19 default login -> 401 +20:37:23 default login -> 429 +20:38:20 default login -> 401 +20:38:23 default login -> 429 +20:39:20 default login -> 401 +20:39:23 default login -> 429 +20:39:45 default login -> 502 +200 with the DEFAULT login (a stranger got in): 0 of 192 polls +2026/09/30 20:36:32 install_hold.go:106: [INFO] [stacks] calibre-web: install HOLD before the first start — only the household reaches [books.enkisfelhom.hu] until the known first login is replaced +2026/09/30 20:37:01 deploy.go:583: [INFO] [stacks] Stack calibre-web deployed successfully (took 28.3s) +2026/09/30 20:37:18 install_hold.go:135: [INFO] [stacks] calibre-web: install hold OPENED by after_install — the app is reached as without a hold + +stranger with the GENERATED password (ADMIN_PASSWORD) -> 404 +after the backup restart — stranger, DEFAULT login: 401 +stranger, a WRONG password: 401 +stranger, no login (the app's own page, the hold is open): 200 diff --git a/documentation/audits/night-rulings-2026-09-30/C/C3-mealie-poll.txt b/documentation/audits/night-rulings-2026-09-30/C/C3-mealie-poll.txt new file mode 100644 index 00000000..5b308fd3 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/C/C3-mealie-poll.txt @@ -0,0 +1,12 @@ +# mealie: stranger POST /api/auth/token with the DEFAULT login, from the deploy press (controller gitea.dooplex.hu/admin/felhom-controller:0.284.0) +deploy: True +20:41:07 default login -> 404 +20:42:08 default login -> 401 +20:42:23 default login -> 423 +200 with the DEFAULT login (a stranger got in): 0 of 97 +stranger, GENERATED password (positive control): 423 +stranger, a WRONG password: 423 +2026/09/30 20:41:07 install_hold.go:106: [INFO] [stacks] mealie: install HOLD before the first start — only the household reaches [recipes.enkisfelhom.hu] until the known first login is replaced +2026/09/30 20:41:52 deploy.go:583: [INFO] [stacks] Stack mealie deployed successfully (took 44.7s) +2026/09/30 20:42:16 install_hold.go:135: [INFO] [stacks] mealie: install hold OPENED by after_install — the app is reached as without a hold + diff --git a/documentation/audits/night-rulings-2026-09-30/D/D1-wger-jwt-probe.txt b/documentation/audits/night-rulings-2026-09-30/D/D1-wger-jwt-probe.txt new file mode 100644 index 00000000..2ac1058f --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/D/D1-wger-jwt-probe.txt @@ -0,0 +1,9 @@ +web login page: 200 +key file: -rw------- +2 +FELHOM_AFTER_INSTALL_OK +app login, RIGHT password: 200 keys=['access_token', 'is_authenticated', 'refresh_token', 'session_token'] +app login, WRONG password: 400 +API with the app token: 200 +after a restart the key file is the same: 2 + Volume probewger_wger_data Removed diff --git a/documentation/audits/night-rulings-2026-09-30/D/D2-wanderer-meili-probe.txt b/documentation/audits/night-rulings-2026-09-30/D/D2-wanderer-meili-probe.txt new file mode 100644 index 00000000..1e3c3f60 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/D/D2-wanderer-meili-probe.txt @@ -0,0 +1,19 @@ + Container wanderer-search Started +wanderer-search getmeili/meilisearch:v1.54.2 Restarting (1) 9 seconds ago +To migrate data between Meilisearch versions, please follow our guide on https://www.meilisearch.com/docs/learn/update_and_migration/updating. +Alternatively, you can set the `--upgrade-db` flag (or the `MEILI_UPGRADE_DB` environment variable) to upgrade the database on startup. +2026-09-30T20:34:11.499060Z ERROR meilisearch: error=Your database version (1.36.0) is incompatible with your current engine version (1.54.2). +To migrate data between Meilisearch versions, please follow our guide on https://www.meilisearch.com/docs/learn/update_and_migration/updating. +Alternatively, you can set the `--upgrade-db` flag (or the `MEILI_UPGRADE_DB` environment variable) to upgrade the database on startup. +Error: Your database version (1.36.0) is incompatible with your current engine version (1.54.2). +To migrate data between Meilisearch versions, please follow our guide on https://www.meilisearch.com/docs/learn/update_and_migration/updating. +Alternatively, you can set the `--upgrade-db` flag (or the `MEILI_UPGRADE_DB` environment variable) to upgrade the database on startup. + Container wanderer-search Started +== with MEILI_UPGRADE_DB=true: +wanderer-search getmeili/meilisearch:v1.54.2 Up 30 seconds (healthy) +2026-09-30T20:34:35.667760Z  INFO HTTP request{method=POST host="wanderer-search:7700" route=/indexes/trails/search [3 +2026-09-30T20:34:50.120229Z  INFO HTTP request{method=GET host="127.0.0.1:7700" route=/health query_parameters +2026-09-30T20:34:59.459524Z  INFO HTTP request{method=GET host="wanderer-search:7700" route=/keys query_parameters +2026-09-30T20:34:59.611097Z  INFO HTTP request{method=POST host="wanderer-search:7700" route=/indexes/trails/search [3 +indexes: ['actors', 'lists', 'trails'] +version: {"commitSha":"2b1d96d0a3ca4ccb86329280002c875c8f5ffb60","commitDate":"unknown","pkgVersion":"1.54.2"} diff --git a/documentation/audits/night-rulings-2026-09-30/D/D3-wanderer-routes-probe.txt b/documentation/audits/night-rulings-2026-09-30/D/D3-wanderer-routes-probe.txt new file mode 100644 index 00000000..99864100 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/D/D3-wanderer-routes-probe.txt @@ -0,0 +1,15 @@ +web / = 200 +signup: ","collectionName":"users","created":"2026-09-30 21:14:57.894Z","emailVisibility":false,"id":"h3wdpfrfz94iv16","name":"","updated":"2026-09-30 21:14:57.894Z","username":"drillw1","verified":false} 200 +login 200 +set-cookie: meilisearch_token=eyJhbGciOiJIUzI1NiIsInR5cCI6Ik +set-cookie: pb_auth=%7B%22token%22%3A%22eyJhbGciOiJIUzI1NiIs +{"record":{"avatar":"","collectionId":"_pb_users_auth_","collectionName":"users","created":"2026-09-30 21:14:57.894Z","email":"drillw1@gate.invalid"," +list create: :["public"],"message":"Required"},{"code":"invalid_type","expected":"array","received":"undefined","path":["trails"],"message":"Required"},{"code":"invalid_type","expected":"string","received":"undefined","path":["author"],"message":"Required"}]} 400 +list get: {"items":[],"page":1,"perPage":30,"totalItems":0,"totalPages":0} 200 +search: {"hits":[],"query":"upgdrilllist1","processingTimeMs":7,"limit":20,"offset":0,"estimatedTotalHits":0,"requestUid":"01a0f42b-ad6c-7573-a7e1-0e03426c6153"} 200 +list create: {"data":{},"message":"Failed to create record.","status":400,"detail":{"data":{},"message":"Failed to create record.","status":400}} 400 +list get: "totalItems":0 200 +search hit: "estimatedTotalHits":0 +search miss: "estimatedTotalHits":0 +no login: {"items":[],"page":1,"perPage":30,"totalItems":0,"totalPages":0} 200 + Network probewanderer_wanderer-internal Removed diff --git a/documentation/audits/night-rulings-2026-09-30/E/E1-demo-boxes-before.txt b/documentation/audits/night-rulings-2026-09-30/E/E1-demo-boxes-before.txt new file mode 100644 index 00000000..ae5d9942 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/E/E1-demo-boxes-before.txt @@ -0,0 +1,12 @@ +## demo-hp guest 9201 2026-09-30T21:10:39Z +gitea.dooplex.hu/admin/felhom-controller:0.283.1 Up 9 hours (healthy) +TYPE TOTAL ACTIVE SIZE RECLAIMABLE +Images 130 20 24.25GB 13.49GB (55%) +130 +/dev/mapper/pve-vm--9201--disk--1 69G 28G 39G 42% /var/lib/docker +## felhom-pve guest 9201 2026-09-30T21:10:42Z +gitea.dooplex.hu/admin/felhom-controller:0.283.1 Up 12 hours (healthy) +TYPE TOTAL ACTIVE SIZE RECLAIMABLE +Images 82 5 6.114GB 3.688GB (60%) +82 +/dev/mapper/pve-vm--9201--disk--1 246G 6.0G 228G 3% /var/lib/docker diff --git a/documentation/audits/night-rulings-2026-09-30/E/E2-floor.txt b/documentation/audits/night-rulings-2026-09-30/E/E2-floor.txt new file mode 100644 index 00000000..b41faec2 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/E/E2-floor.txt @@ -0,0 +1,7 @@ +# floor 2026-09-30T21:10:55Z +impact: {"below":4,"valid":true,"version":"0.284.2"} +HTTP/1.1 303 See Other +Location: /configuration?flash=floor_set +2026/09/30 23:10:55 [INFO] Global controller-version floor set to "0.284.2" (declared MinAgent "0.131.0") +2026/09/30 23:10:57 [INFO] managed floor SERVED for demo-felhom: floor 0.284.2, agent requirement "0.131.0" from declared (golden 0.283.1) +2026/09/30 23:10:58 [INFO] managed floor SERVED for demo-hp: floor 0.284.2, agent requirement "0.131.0" from declared (golden 0.283.1) diff --git a/documentation/audits/night-rulings-2026-09-30/E/E3-demo-boxes-after-floor.txt b/documentation/audits/night-rulings-2026-09-30/E/E3-demo-boxes-after-floor.txt new file mode 100644 index 00000000..9f126307 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/E/E3-demo-boxes-after-floor.txt @@ -0,0 +1,5 @@ +## demo-hp 2026-09-30T21:13:55Z +gitea.dooplex.hu/admin/felhom-controller:0.284.2 Up 2 minutes (healthy) +2026/09/30 21:11:12 updater.go:102: [DEBUG] [selfupdate] SetFloor: floor "" → "0.284.2" +## felhom-pve 2026-09-30T21:13:57Z +gitea.dooplex.hu/admin/felhom-controller:0.284.2 Up 2 minutes (healthy) diff --git a/documentation/audits/night-rulings-2026-09-30/E/E4-demo-boxes-sweep.txt b/documentation/audits/night-rulings-2026-09-30/E/E4-demo-boxes-sweep.txt new file mode 100644 index 00000000..3e5e5a7d --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/E/E4-demo-boxes-sweep.txt @@ -0,0 +1,10 @@ +## demo-hp 2026-09-30T21:15:31Z +2026/09/30 21:14:40 image_retention.go:364: [INFO] [stacks] image retention (one-time): deleted 24 image(s). docker disk before: Images 24.29GB 13.52GB (55%) | after: Images 13.48GB 3.022GB (22%) +2 +Images 107 20 13.48GB 3.022GB (22%) +/dev/mapper/pve-vm--9201--disk--1 69G 16G 50G 25% /var/lib/docker +## felhom-pve 2026-09-30T21:15:33Z +2026/09/30 21:14:03 [INFO] [stacks] image retention (one-time): deleted 1 image(s). docker disk before: Images 6.151GB 3.725GB (60%) | after: Images 6.068GB 3.641GB (60%) +2 +Images 82 5 6.068GB 3.641GB (60%) +/dev/mapper/pve-vm--9201--disk--1 246G 6.0G 228G 3% /var/lib/docker diff --git a/documentation/audits/night-rulings-2026-09-30/README.md b/documentation/audits/night-rulings-2026-09-30/README.md new file mode 100644 index 00000000..1fcfab26 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/README.md @@ -0,0 +1,15 @@ +# The operator's two rulings built — 2026-09-30 (late evening) + +Report and Part table: `felhom.eu/REPORT-night-rulings-2026-09-30.md`. Golden: `documentation/tests/golden-0.284.2-2026-09-30/`. + +| folder | what | +|---|---| +| `A/A0-spike.md` | Part A spike: what the writer and gates had to accept and refuse, read before building | +| `A/A1-decoys-red.txt` | the 8 re-test decoys with the rules switched off (red) | +| `A/e2e/` | end to end on 9202: `E1*` the drill's month-ago state, `bench/` the re-test (docmost), `box/` the box half (the leg's own lines, the badge before/after), `E2-writer.txt` the real writer into a scratch copy + both gates with a positive and a negative registry control; `*outline-attempt*` the first attempt, stopped by outline's fixture (R-744) | +| `B/` | image retention: `B1` red-proofs; `B2`–`B8` 9202 before, the deploys, the one-time sweeps (v1 blind to untagged images, v2 26.6 → 5.7 GB), the removes | +| `C/` | the install hold: `C1` red-proofs; `C2` calibre-web and `C3` mealie polled as a stranger | +| `D/` | `D1` wger's key on the bench; `D2` meilisearch v1.36 → v1.54; `D3` wanderer's routes with the bench override | +| `E/` | the demo boxes before/after the floor, the floor, their one-time sweeps | +| `T/` | teardown: 9202 back on live, the bench destroyed, the drill reset | +| `tools/` | `walk.py`, `repoint.py` (this folder's paths), `r741poll.py`, `r741mealie.py` | diff --git a/documentation/audits/night-rulings-2026-09-30/T/T1-9202-repoint-live.txt b/documentation/audits/night-rulings-2026-09-30/T/T1-9202-repoint-live.txt new file mode 100644 index 00000000..746d4ddd --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/T/T1-9202-repoint-live.txt @@ -0,0 +1,16 @@ +git: + branch: main + repo_url: https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git + sync_interval: 15m + token: + username: "" +hub: +0 + +felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.284.2 +filebrowser gtstef/filebrowser:1.3.3-stable +paperless-webserver ghcr.io/paperless-ngx/paperless-ngx:2.20.15 +paperless-postgres postgres:18-alpine +paperless-redis redis:7-alpine +traefik traefik:v3.6.7 +0 diff --git a/documentation/audits/night-rulings-2026-09-30/T/T2-bench-destroy.txt b/documentation/audits/night-rulings-2026-09-30/T/T2-bench-destroy.txt new file mode 100644 index 00000000..8e75d32e --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/T/T2-bench-destroy.txt @@ -0,0 +1,7 @@ +## 2026-09-30T21:22:12Z bench destroy +upgrade-harness +purging CT 9401 from related configurations.. +template-removed +VMID Status Lock Name +9201 running demo-hp +9202 running demo-hp-scratch diff --git a/documentation/audits/night-rulings-2026-09-30/T/T3-drill-reset.txt b/documentation/audits/night-rulings-2026-09-30/T/T3-drill-reset.txt new file mode 100644 index 00000000..d1bd8582 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/T/T3-drill-reset.txt @@ -0,0 +1,8 @@ +# drill reset 2026-09-30T21:22:29Z +drill before: 429563b; live main: 6a3ead9 +429563b DRILL docmost: re-test of the same tag {'docmost-redis': 'sha256:858f009f9709'} (box proof) +1189d86 DRILL e2e (decision 52): docmost's head says redis:7-alpine was tested at an OLDER digest c35af3bb +995a8e1 DRILL e2e (decision 52): outline's head says redis:7-alpine was tested at an OLDER digest c35af3bb (the month-ago state) +remote: . Processing 1 references +drill after: 6a3ead9 +image lines IDENTICAL diff --git a/documentation/audits/night-rulings-2026-09-30/bench/B1-bench-create.txt b/documentation/audits/night-rulings-2026-09-30/bench/B1-bench-create.txt new file mode 100644 index 00000000..81a980c1 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/bench/B1-bench-create.txt @@ -0,0 +1,25 @@ ++ date -u +Wed Sep 30 20:16:30 UTC 2026 ++ pct list +VMID Status Lock Name +9201 running demo-hp +9202 running demo-hp-scratch ++ free -g ++ head -2 + total used free shared buff/cache available +Mem: 29 5 13 0 11 23 ++ pveam download local debian-13-standard_13.6-1_amd64.tar.zst ++ tail -1 +download of 'http://download.proxmox.com/images/system/debian-13-standard_13.6-1_amd64.tar.zst' to '/var/lib/vz/template/cache/debian-13-standard_13.6-1_amd64.tar.zst' finished ++ pct create 9401 local:vztmpl/debian-13-standard_13.6-1_amd64.tar.zst --hostname upgrade-harness --cores 6 --memory 10240 --swap 0 --rootfs nvme-scratch:60 --net0 name=eth0,bridge=vmbr0,ip=dhcp --unprivileged 1 --features nesting=1,keyctl=1 --onboot 0 ++ tail -1 +done: SHA256:opoIy9f8z7d4CvLizMbawjiKWiRczXKM0kWtPrPm15M root@upgrade-harness ++ pct start 9401 ++ sleep 15 ++ pct exec 9401 -- bash -c 'apt-get update -qq >/dev/null 2>&1; DEBIAN_FRONTEND=noninteractive apt-get install -y -qq docker.io docker-compose python3 python3-yaml curl postgresql-client sqlite3 >/dev/null 2>&1; docker --version; docker compose version; docker network create traefik-public >/dev/null && echo net; free -m | head -3' +Docker version 26.1.5+dfsg1, build a72d7cd +Docker Compose version 2.26.1-4 +net + total used free shared buff/cache available +Mem: 10240 59 8533 0 1646 10180 +Swap: 0 0 0 diff --git a/documentation/audits/night-rulings-2026-09-30/tools/r741mealie.py b/documentation/audits/night-rulings-2026-09-30/tools/r741mealie.py new file mode 100644 index 00000000..317f0756 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/tools/r741mealie.py @@ -0,0 +1,33 @@ +#!/usr/bin/env python3 +"""R-741 on mealie as a STRANGER (no session, no gate cookie): POST /api/auth/token with the DEFAULT login once a +second from the deploy press; then, once the app answers, the GENERATED password (positive control) and a wrong one.""" +import subprocess, threading, time, json +from datetime import datetime, timezone +import walk as w +w.login() +def token(user, pw): + r = subprocess.run(["curl","-sk","-o","/dev/null","-w","%{http_code}","--max-time","8","-H",f"Host: recipes.{w.DOMAIN}", + "--data-urlencode",f"username={user}","--data-urlencode",f"password={pw}",f"{w.BASE}/api/auth/token"],capture_output=True,text=True) + return r.stdout.strip() +res, stop = [], threading.Event() +def poll(): + while not stop.is_set(): + res.append((datetime.now(timezone.utc).strftime("%H:%M:%S"), token("changeme@example.com","MyPassword"))); time.sleep(1) +print("# mealie: stranger POST /api/auth/token with the DEFAULT login, from the deploy press (controller %s)" % w.guest("docker inspect felhom-controller --format {{.Config.Image}}").strip()) +t = threading.Thread(target=poll, daemon=True); t.start() +print("deploy:", w.deploy("mealie", "recipes")) +for _ in range(120): + time.sleep(2) + if "hold OPENED" in w.guest("docker logs --since 10m felhom-controller 2>&1 | grep 'mealie: install hold OPENED'"): + break +time.sleep(30); stop.set(); t.join() +prev = None +for ts, c in res: + if c != prev: + print(ts, "default login ->", c); prev = c +print("200 with the DEFAULT login (a stranger got in):", sum(1 for _, c in res if c == "200"), "of", len(res)) +gen = (w.GENERATED.get("mealie") or {}).get("ADMIN_PASSWORD", "") +print("stranger, GENERATED password (positive control):", token("changeme@example.com", gen)) +print("stranger, a WRONG password:", token("changeme@example.com", "wrong-" + str(time.time()))) +print(w.guest("docker logs --since 10m felhom-controller 2>&1 | grep -E 'mealie.*(install HOLD|hold OPENED|after_install .*done|deployed successfully)' | cut -c1-200")) +w.remove("mealie") diff --git a/documentation/audits/night-rulings-2026-09-30/tools/r741poll.py b/documentation/audits/night-rulings-2026-09-30/tools/r741poll.py new file mode 100644 index 00000000..03929bd7 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/tools/r741poll.py @@ -0,0 +1,31 @@ +#!/usr/bin/env python3 +"""r741poll.py — R-741's method (A3 of more-night-apps), as a STRANGER: plain +curl through traefik with the app's Host, NO dashboard session and NO gate cookie, once a second from the deploy press +for 150 s, with the default login and (after the install) the generated one. Read-only except the install it makes.""" +import sys, time, threading, subprocess, json +from datetime import datetime, timezone +import walk as w +app, sub, dflt, path = sys.argv[1:5] +w.login() +def stranger(cred): + r = subprocess.run(["curl", "-sk", "-o", "/dev/null", "-w", "%{http_code}", "--max-time", "5", "-H", f"Host: {sub}.{w.DOMAIN}", + "-u", cred, f"{w.BASE}{path}"], capture_output=True, text=True) + return r.stdout.strip() +res, stop = [], threading.Event() +def poll(): + while not stop.is_set(): + res.append((datetime.now(timezone.utc).strftime("%H:%M:%S"), stranger(dflt))); time.sleep(1) +print(f"# {app}: stranger polls of {path} with the DEFAULT login, from the deploy press (controller {w.guest('docker inspect felhom-controller --format {{.Config.Image}}').strip()})") +t = threading.Thread(target=poll, daemon=True); t.start() +print("deploy:", w.deploy(app, sub)) +time.sleep(150); stop.set(); t.join() +prev = None +for ts, c in res: + if c != prev: + print(ts, "default login ->", c); prev = c +print("200 with the DEFAULT login (a stranger got in):", sum(1 for _, c in res if c == "200"), "of", len(res), "polls") +print(w.guest(f"docker logs --since 5m felhom-controller 2>&1 | grep -E '{app}.*(install HOLD|hold OPENED|after_install .*done|deployed successfully)' | cut -c1-220")) +gen = (w.GENERATED.get(app) or {}) +for k, v in gen.items(): + if "PASS" in k: + print(f"stranger with the GENERATED password ({k}) -> {stranger('admin:' + v)}") diff --git a/documentation/audits/night-rulings-2026-09-30/tools/repoint.py b/documentation/audits/night-rulings-2026-09-30/tools/repoint.py new file mode 100644 index 00000000..43f19094 --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/tools/repoint.py @@ -0,0 +1,60 @@ +#!/usr/bin/env python3 +"""Point guest 9202 at the drill catalog (and a 90 s health timeout), or restore the saved config. + +`09` §6.5: `git.repo_url` alone is INERT (R-615) — the cache dir must go too. The saved copy is +`controller.yaml.pre-rulings0930` (NOT the older `.pre-28`, which a restore must never pick up). +""" +import re, sys, io +sys.path.insert(0, '.') +import walk as w + +VOL = "/var/lib/docker/volumes/felhom-controller-data/_data" +DRILL_REPO = "https://gitea.dooplex.hu/admin/app-catalog-drill.git" + + +def creds(): + for l in io.open("/home/kisfenyo/.git-credentials").read().strip().split("\n"): + m = re.match(r'https://(admin):([^@]+)@gitea\.dooplex\.hu', l) + if m: + return m.group(1), m.group(2) + raise SystemExit("no admin credential") + + +def to_drill(): + u, t = creds() + print(w.guest(f""" +set -e +test -f {VOL}/controller.yaml.pre-rulings0930 || cp -p {VOL}/controller.yaml {VOL}/controller.yaml.pre-rulings0930 +python3 - <<'PY' +import re +p = "{VOL}/controller.yaml" +s = open(p).read() +s = re.sub(r'(^\\s+repo_url: ).*$', r'\\g<1>{DRILL_REPO}', s, count=1, flags=re.M) +s = re.sub(r'(^git:(?:\\n\\s+.*)*?\\n\\s+token: ).*$', r'\\g<1>"{t}"', s, count=1, flags=re.M) +s = re.sub(r'(^git:(?:\\n\\s+.*)*?\\n\\s+username: ).*$', r'\\g<1>"{u}"', s, count=1, flags=re.M) +if not re.search(r'^update:', s, re.M): + s += "update:\\n health_timeout: 90s\\n" +open(p, "w").write(s) +PY +rm -rf {VOL}/catalog-cache {VOL}/data/catalog-cache +docker restart felhom-controller >/dev/null +sleep 15 +grep -A6 '^git:' {VOL}/controller.yaml | sed 's/token:.*/token: /' +grep -A2 '^update:' {VOL}/controller.yaml +""")) + + +def restore(): + print(w.guest(f""" +set -e +cp -p {VOL}/controller.yaml.pre-rulings0930 {VOL}/controller.yaml +rm -rf {VOL}/catalog-cache {VOL}/data/catalog-cache +docker restart felhom-controller >/dev/null +sleep 15 +grep -A6 '^git:' {VOL}/controller.yaml | sed 's/token:.*/token: /' +grep -c '^update:' {VOL}/controller.yaml || true +""")) + + +if __name__ == "__main__": + to_drill() if sys.argv[1] == "drill" else restore() diff --git a/documentation/audits/night-rulings-2026-09-30/tools/walk.py b/documentation/audits/night-rulings-2026-09-30/tools/walk.py new file mode 100644 index 00000000..50d900ea --- /dev/null +++ b/documentation/audits/night-rulings-2026-09-30/tools/walk.py @@ -0,0 +1,560 @@ +#!/usr/bin/env python3 +"""walk.py — ONE app's full update walk on guest 9202, through the product's own endpoints. + +EVIDENCE, NOT PRODUCT. It presses exactly the buttons a person presses: + POST /api/stacks//deploy · POST /api/sync · POST /api/stacks/rescan + POST /api/stacks//update · POST /api/stacks//remove +and reads GET /api/stacks/. No controller code exists for it. + +The walk, per `09` §6.4 and the update-night brief §4: + 1 deploy from the DRILL catalog at the LIVE pin + 2 seed through the app's OWN front door (R-156: never a volume, never SQL) + 3 read the seed back <- control C1; a fixture that cannot prove itself proves nothing + 4 „Mentés most" + 5 commit the real one-step bump to the DRILL repo, sync, rescan, read the badge in BOTH languages + 6 press the guarded Update, record every phase with timestamps + 7 read the seed back through the front door + 8 the four version observables side by side + 9 write the verdict record in `09`'s JSON shape + +`inconclusive` is a first-class verdict and is NEVER collapsed into `failed`. +""" +import argparse, json, os, re, subprocess, sys, time +from datetime import datetime, timezone + +SC = os.environ.get('SC', '/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/2930eec5-3257-446d-80bf-8921da206cd8/scratchpad') +EV = os.environ.get('EV', '/mnt/5_hdd/felhom.eu/git/felhom.eu/documentation/audits/night-rulings-2026-09-30') +DRILL = "/mnt/5_hdd/felhom.eu/drill/app-catalog-drill" +# GUEST=9201 selects demo-hp's hub-enabled guest (the mail proof); default 9202, the scratch guest. +GUEST = os.environ.get("GUEST", "9202") +BASE = os.environ.get("BASE") or {"9202": "https://192.168.0.114", "9201": "https://192.168.0.155"}[GUEST] +DOMAIN = os.environ.get("DOMAIN", "enkisfelhom.hu") +HOSTHDR = f"Host: felhom.{DOMAIN}" +HP = "demo-hp" + +LOG = [] + + +def say(*a): + line = " ".join(str(x) for x in a) + ts = datetime.now().strftime("%H:%M:%S") + print(f"{ts} {line}", flush=True) + LOG.append(f"{ts} {line}") + + +def sh(args, timeout=300, inp=None): + try: + return subprocess.run(args, capture_output=True, text=True, timeout=timeout, input=inp) + except (subprocess.TimeoutExpired, OSError) as e: + return subprocess.CompletedProcess(args, 124, "", f"{e}") + + +def guest(script, timeout=600): + """Run a bash script inside guest 9202. Piped as a file — never as an argument (quoting).""" + # ONE TEMP FILE PER CALL (night 2026-09-23): the shared /tmp/w.sh swapped scripts under + # two concurrent walks (memory: guest-helper-shares-one-tmp-file). + import secrets as _s + t = f"/tmp/w{GUEST}-{os.getpid()}-{_s.token_hex(4)}.sh" + r = sh(["ssh", "-o", "ConnectTimeout=20", "-o", "StrictHostKeyChecking=accept-new", HP, + f"export LC_ALL=C; cat > {t}; pct push {GUEST} {t} {t} >/dev/null 2>&1; " + f"pct exec {GUEST} -- bash {t}; pct exec {GUEST} -- rm -f {t}; rm -f {t}"], + timeout=timeout, inp=script) + return r.stdout or "" + + +def login(): + pw = open(f"{SC}/.ctlpw").read().strip() + sh(["curl", "-sk", "-D", f"{SC}/hdr{os.getpid()}.txt", "-o", "/dev/null", "-H", HOSTHDR, + "-X", "POST", "--data-urlencode", f"password={pw}", f"{BASE}/login"]) + h = open(f"{SC}/hdr{os.getpid()}.txt").read() + m = re.search(r"felhom_session=[A-Za-z0-9._-]+", h, re.I) + if not m: + sys.exit("login failed: no session cookie") + open(f"{SC}/sess{os.getpid()}.txt", "w").write(m.group(0)) + r = sh(["curl", "-sk", "-L", "-H", HOSTHDR, "-H", f"Cookie: {m.group(0)}", f"{BASE}/"]) + c = re.search(r' the felhom_gate cookie the household's browser would hold (setup gate, v0.280.0) + + +def _loc(out): + m = re.search(r"(?im)^location:\s*(\S+)", out or "") + return m.group(1) if m else "" + + +def gate_cookie(sub): + """Pass the setup gate (`09` decision 46) the way the HOUSEHOLD does: the app host redirects to the + dashboard's /__gate/start, which (with the dashboard session) redirects back to the app host's + /__felhom_gate/cb, which sets `felhom_gate`. Never printed.""" + import urllib.parse + sess = open(f"{SC}/sess{os.getpid()}.txt").read().strip() + r = sh(["curl", "-sk", "-D", "-", "-o", "/dev/null", "-H", "Accept: text/html", "-H", f"Host: {sub}.{DOMAIN}", f"{BASE}/"]) + loc = _loc(r.stdout) + if "/__gate/start" not in loc: + GATE[sub] = "" + return "" # not gated (open, or no gate for this app) + u = urllib.parse.urlsplit(loc) + r = sh(["curl", "-sk", "-D", "-", "-o", "/dev/null", "-H", "Accept: text/html", "-H", f"Host: {u.hostname}", "-H", f"Cookie: {sess}", + f"{BASE}{u.path}?{u.query}"]) + loc = _loc(r.stdout) + u = urllib.parse.urlsplit(loc) + if "/__felhom_gate/cb" not in u.path: + say(f" gate: the dashboard did not hand back a callback for {sub} ({loc[:80]})") + return "" + r = sh(["curl", "-sk", "-D", "-", "-o", "/dev/null", "-H", "Accept: text/html", "-H", f"Host: {u.hostname}", f"{BASE}{u.path}?{u.query}"]) + m = re.search(r"(?im)^set-cookie:\s*(felhom_gate=[^;\r\n]+)", r.stdout or "") + GATE[sub] = m.group(1) if m else "" + say(f" gate: {sub} is gated — passed as the household (cookie {'set' if GATE[sub] else 'NOT set'})") + return GATE[sub] + + +def app_curl(sub, path, *extra, method=None, data=None, timeout=45, _retry=True): + """A call to the APP's own front door on 9202 — the household's route, not ours. Carries the setup + gate's cookie when the app is gated, merged into a fixture's own Cookie header (never a second one).""" + raw = list(extra) + gc = GATE[sub] if sub in GATE else gate_cookie(sub) + ext = list(raw) + if gc: + merged = False + for i, a in enumerate(ext): + if isinstance(a, str) and a.lower().startswith("cookie:") and i > 0 and ext[i - 1] == "-H": + ext[i] = a + "; " + gc + merged = True + if not merged: + ext = ["-H", f"Cookie: {gc}"] + ext + args = ["curl", "-sSk", "--max-time", str(timeout), "-H", f"Host: {sub}.{DOMAIN}", + "-w", "\n%{http_code} %{redirect_url}"] + if method: + args += ["-X", method] + if data is not None: + args += ["--data-binary", "@-"] + args += ext + [f"{BASE}{path}"] + r = sh(args, timeout=timeout + 30, inp=data) + body, _, tail = (r.stdout or "").rpartition("\n") + code, _, redir = tail.strip().partition(" ") + if _retry and "/__gate/start" in redir: + GATE.pop(sub, None) # the gate cookie expired or was never taken — log in as the household again + return app_curl(sub, path, *raw, method=method, data=data, timeout=timeout, _retry=False) + return r.returncode, code.strip(), body + + +def stack(name): + _, d = ctl("GET", f"/api/stacks/{name}") + return (d.get("data") or {}) if isinstance(d, dict) else {} + + +def wait_app(sub, path="/", want=("200", "302", "303", "401", "403"), tries=60, delay=5): + """Settling says the container runs; this says the APP answers. Not the same thing.""" + last = None + for _ in range(tries): + rc, code, _ = app_curl(sub, path, timeout=15) + last = (rc, code) + if rc == 0 and code in want: + return True + time.sleep(delay) + say(f" app never answered on {sub}{path} (last rc={last[0]} code={last[1]})") + return False + + +# ------------------------------------------------------------------ the walk + + +DRIVE = "/mnt/felhom-drives/scratch_hdd/userdata" + +# What THIS run generated for a deploy, per app. Deploy secrets are ENCRYPTED AT REST in +# `app.yaml` (`ENC:…`), which is right and which means a fixture cannot read an app's admin +# password back off the box — the household sees it once. So the value the harness itself +# generated is kept here for the life of the run, and nowhere else. +GENERATED = {} + + +def deploy_values(name, sub): + """Fill EVERY required deploy field the way the wizard would, by asking the box what this app + asks for — `GET /api/stacks//deploy-fields` — instead of assuming DOMAIN+SUBDOMAIN. + + Measured 2026-09-21: three apps in one batch refused at the deploy with a correct 400 because + a required field was absent — `HDD_PATH` (navidrome, audiobookshelf) and an admin password + (grafana). The refusals happen BEFORE anything is created (`deploy.go:324`), which is the only + reason this was safe to discover by running it (live-probes rule). + + A `path` field must name a directory that ALREADY EXISTS (`deploy.go:330`), so one is made on + the scratch drive first — the same act the drive browser performs for a household. + """ + code, d = ctl("GET", f"/api/stacks/{name}/deploy-fields") + fields = (((d.get("data") or {}).get("metadata") or {}).get("deploy_fields")) or [] + values = {"DOMAIN": DOMAIN, "SUBDOMAIN": sub} + made = [] + for f in fields: + ev, ty = f.get("env_var"), f.get("type") + if ev in values: + continue + # `type: password` is MANDATORY whatever `required` says — `deploy.go:305-312` refuses + # when the caller sends none, deliberately ("the user needs to know their password"), + # while `.felhom.yml` declares `required: false` and the API serves that verbatim. A + # caller that trusts the contract gets a 400. Measured tonight on grafana; filed. + if not f.get("required") and ty != "password": + continue # the controller generates the optional secrets itself + if ty == "path": + p = f"{DRIVE}/{name}" + values[ev] = p + made.append(p) + elif ty in ("secret", "password"): + import secrets as _s + values[ev] = "Drill-" + _s.token_hex(12) + GENERATED.setdefault(name, {})[ev] = values[ev] + elif f.get("default"): + values[ev] = f["default"] + else: + values[ev] = f"drill-{name}" + if made: + guest("mkdir -p " + " ".join(made) + "; ls -ld " + " ".join(made)) + say(f" [1] made the drive paths this app requires: {made}") + extra = [k for k in values if k not in ("DOMAIN", "SUBDOMAIN")] + if extra: + say(f" [1] required fields filled beyond DOMAIN/SUBDOMAIN: {extra}") + return values + + +def deploy(name, sub, extra_values=None): + st = stack(name) + if st.get("deployed"): + say(f" [1] {name} already deployed — reusing") + return True + values = deploy_values(name, sub) + if extra_values: + values.update(extra_values) + body = {"values": values} + if os.environ.get("KEPT"): # decision 36: the household's answer when the drive holds old data ("fresh" moves it aside, deletes nothing) + body["kept_data"] = os.environ["KEPT"] + code, d = ctl("POST", f"/api/stacks/{name}/deploy", body) + say(f" [1] deploy -> {code} {str(d)[:120]}") + if code != "202": + return False + # WAIT FOR `deployed`, NOT FOR `running`. Measured 2026-09-21 on tandoor: docker reported the + # container `healthy` while the controller's own state read `unhealthy` — a gate on `running` + # alone therefore times out on an app that is up. The state is RECORDED rather than required; + # the real gate is the fixture's own `wait_app`, which asks whether the APP answers. + seen = None + for _ in range(90): + time.sleep(5) + st = stack(name) + seen = st.get("state") + # `deployed` alone is NOT enough and `state` alone is NOT right. Measured 2026-09-21: + # tandoor reads `unhealthy` while serving (R-618), so gating on "running" hangs; and romm + # read `deployed=True, state=degraded, pinned_images=None` twenty seconds in, i.e. the + # deploy had not finished writing app.yaml. The PIN is the deploy's own completion mark + # (`runComposeDeploy` writes it), so that is what to wait for. + pins = (st.get("app_config") or {}).get("pinned_images") + if st.get("deployed") and pins and seen in ("running", "unhealthy", "degraded"): + say(f" [1] deployed, controller state={seen}, " + f"pinned={(st.get('app_config') or {}).get('pinned_images')}") + if seen != "running": + say(f" [1] NOTE: the controller's own state is {seen!r}, not 'running' — recorded, " + f"not treated as a failure; the fixture's front-door wait is the real gate") + return True + say(f" [1] never became deployed (last controller state={seen!r})") + return False + + +def backup_now(name): + """R-648 (2026-09-23): NO whole-box „Mentés most" from a drill, ever. + + `POST /api/backup/run` is the only backup endpoint and it is WHOLE-BOX: on 9201 it stopped and + restarted 9 of 10 standing apps twice, and on 9202 it broke a deploy in flight (R-634). The product + has NO per-app backup endpoint (router.go: /backup/run, /backup/tier2 only); the per-app backup + exists only inside the guarded update, whose `backing-up` phase calls RunAppBackupNow for the one + app. So this presses nothing: the update takes the throwaway app's own backup, and says so in its + phase list. A seed written "after the backup" is therefore written before the update's own backup + — the undo's last-second copy is still the one that must bring it back.""" + say(f" [4] backup press SKIPPED for {name} (R-648: whole-box only; the update's backing-up phase backs up {name} alone)") + return None + +def drill_bump(app, frm, to, service_hint=None): + """Serialised across concurrent walks: one git working tree, one lock.""" + import fcntl + with open(f"{SC}/drill.lock", "w") as lk: + fcntl.flock(lk, fcntl.LOCK_EX) + sh(["git", "-C", DRILL, "pull", "-q", "--rebase", "origin", "main"], timeout=120) + return _drill_bump(app, frm, to, service_hint) + + +def _drill_bump(app, frm, to, service_hint=None): + """Commit the edge to the DRILL repo. catalog_since set by hand (the drill repo has no gates). + + `frm`/`to` may be comma-separated lists of the SAME length: an app whose own version lives in + two images (adventurelog's backend and frontend) moves both in one edge, while its engine + sidecar stays where it is — `09` §3b Q3's rule is per SERVICE, and an app-half edge must move + every service that carries the app's own version and no others. + """ + comp = f"{DRILL}/templates/{app}/docker-compose.yml" + fy = f"{DRILL}/templates/{app}/.felhom.yml" + s = open(comp).read() + froms = [x.strip() for x in frm.split(",") if x.strip()] + tos = [x.strip() for x in to.split(",") if x.strip()] + if len(froms) != len(tos): + say(f" [5] from/to lists differ in length: {froms} vs {tos}") + return None + for f1, t1 in zip(froms, tos): + if f"image: {f1}" not in s: + say(f" [5] FROM ref not found in compose: {f1}") + return None + s = s.replace(f"image: {f1}", f"image: {t1}") + open(comp, "w").write(s) + f = open(fy).read() + today = datetime.now().strftime("%Y-%m-%d") + f = re.sub(r'^catalog_since:.*$', f'catalog_since: "{today}"', f, count=1, flags=re.M) + open(fy, "w").write(f) + sh(["git", "-C", DRILL, "add", "-A"]) + sh(["git", "-C", DRILL, "commit", "-q", "-m", f"DRILL {app}: {frm} -> {to}"]) + r = sh(["git", "-C", DRILL, "push", "-q", "origin", "main"], timeout=120) + h = sh(["git", "-C", DRILL, "rev-parse", "--short=12", "HEAD"]).stdout.strip() + say(f" [5] drill commit {h}: {app} {frm} -> {to} (push rc={r.returncode})") + return h + + +def sync_rescan(expect_app=None, expect_ref=None, tries=12, delay=5): + """Sync, rescan, and — when told what to expect — WAIT FOR THE BADGE TO CATCH UP. + + R-607: `POST /api/sync` answers "nincs valtozas" while the catalog HAS moved, and + `catalog_images` stays stale until a separate rescan. Tonight showed the rescan alone is not + enough either: mealie's badge read "Naprakesz" seconds after its bump was pushed, and the + Update that followed moved nothing and still reported "Frissitve". So when the caller knows + which reference should appear, this polls for it and SAYS HOW LONG IT TOOK — which is the + NUMBER R-607 asks for and has never had. + """ + t0 = time.time() + ctl("POST", "/api/sync") + time.sleep(2) + ctl("POST", "/api/stacks/rescan") + time.sleep(2) + if not expect_app or not expect_ref: + return None + for i in range(tries): + cat = stack(expect_app).get("catalog_images") or {} + if expect_ref in cat.values(): + waited = round(time.time() - t0, 1) + if i: + say(f" [sync] the badge needed {waited}s and {i+1} sync+rescan rounds to catch up " + f"to {expect_ref} — R-607's window, measured") + return waited + time.sleep(delay) + ctl("POST", "/api/sync") + time.sleep(1) + ctl("POST", "/api/stacks/rescan") + say(f" [sync] the badge NEVER caught up to {expect_ref} in {round(time.time()-t0,1)}s — " + f"catalog_images = {stack(expect_app).get('catalog_images')}") + return None + + +def badges(name): + out = {} + for lang, suffix in (("hu", ""), ("en", "?lang=en")): + h = page(f"/apps/{name}{suffix}") + m = re.findall(r']*title="([^"]*)"[^>]*>([^<]*)<', h) + out[lang] = [{"title": a.strip(), "text": b.strip()} for a, b in m][:3] + return out + + +def press_update(name, poll=1.0, cap_s=1800): + code, d = ctl("POST", f"/api/stacks/{name}/update") + say(f" [6] Update -> {code} {str(d)[:220]}") + if code not in ("202", "200"): + return {"accepted": False, "http": code, "refusal": d, "phases": [], "duration_s": 0} + phases, seen, t0 = [], None, time.time() + while time.time() - t0 < cap_s: + st = stack(name) + ph = st.get("update_phase") + if ph != seen: + seen = ph + rec = {"t": round(time.time() - t0, 1), "phase": ph, + "label": st.get("update_phase_label"), "updating": st.get("updating"), + "error": st.get("update_error"), "hold": st.get("hold_reason")} + phases.append(rec) + say(f" +{rec['t']:>6.1f}s phase={ph} label={rec['label']} " + f"err={rec['error']} hold={rec['hold']}") + if not st.get("updating") and ph in ("done", "failed", "undone", None) and time.time() - t0 > 3: + break + time.sleep(poll) + st = stack(name) + return {"accepted": True, "http": code, "phases": phases, + "duration_s": round(time.time() - t0, 1), + "final_phase": st.get("update_phase"), "update_error": st.get("update_error"), + "hold_reason": st.get("hold_reason"), "state": st.get("state")} + + +def observables(name): + st = stack(name) + ac = st.get("app_config") or {} + live = guest(f""" +grep -E '^\\s+image:' /opt/docker/stacks/{name}/docker-compose.yml 2>/dev/null | sed 's/^ *//' +echo '---inspect---' +for c in $(docker ps -a --filter label=com.docker.compose.project={name} --format '{{{{.Names}}}}'); do + echo -n "$c "; docker inspect "$c" --format '{{{{.Config.Image}}}} running={{{{.State.Running}}}} restarts={{{{.RestartCount}}}}' +done +""") + a, _, b = live.partition("---inspect---") + return { + "pinned_images": ac.get("pinned_images"), + "installed_images": {k: (v.get("ref") if isinstance(v, dict) else v) + for k, v in (ac.get("installed_images") or {}).items()}, + "catalog_images": st.get("catalog_images"), + "live_compose_image_lines": [x for x in a.strip().splitlines() if x.strip()], + "docker_inspect": [x for x in b.strip().splitlines() if x.strip()], + } + + +def app_logs(name, lines=400): + """The app's own container log, DECODED. The endpoint answers a JSON envelope whose `logs` is + one string with escaped newlines — a scan over the envelope sees a single enormous line and + finds nothing, which reads exactly like "the app printed no migration line" and is not. R-96 + rule 3 in a new place: an absent line is not evidence when the instrument cannot see lines.""" + code, d = ctl("GET", f"/api/stacks/{name}/logs?lines={lines}") + if isinstance(d, dict): + data = d.get("data") + if isinstance(data, dict) and isinstance(data.get("logs"), str): + return data["logs"] + if isinstance(d.get("_raw"), str): + return d["_raw"] + return str(d) + + +def write_verdict(rec, appdir): + os.makedirs(appdir, exist_ok=True) + p = os.path.join(appdir, "verdict.json") + json.dump(rec, open(p, "w"), indent=2, ensure_ascii=False) + say(f" [9] verdict {rec['verdict']} -> {p}") + + +def remove(name): + """Remove through the PRODUCT, never `docker rm` (live-probes rule). The remove endpoint + refuses a running stack — `409 still running` — so the stop is part of the act, not a tidy-up.""" + c1, d1 = ctl("POST", f"/api/stacks/{name}/stop") + say(f" [X] stop -> {c1} {str(d1)[:100]}") + for _ in range(24): + time.sleep(5) + if stack(name).get("state") != "running": + break + code, d = ctl("POST", f"/api/stacks/{name}/remove", + {"remove_hdd_data": True, "remove_backups": True}) + say(f" [X] remove (with drive data) -> {code} {str(d)[:160]}") + if code == "409": + # R-442's fail-closed guard: when the storage subsystem cannot RESOLVE the app's drive + # path, the removal is REFUSED and the app is kept rather than half-deleted. On guest 9202 + # `/api/disks` answers `agent not configured`, so every app deployed with an HDD_PATH hits + # this. The household's other choice — remove the app, KEEP the data — is accepted, and the + # harness takes it, then tidies its own directory by name at teardown. + say(" [X] refused because the drive path cannot be resolved (R-442, fail-closed and right)" + " — removing the app and KEEPING the drive data instead") + code, d = ctl("POST", f"/api/stacks/{name}/remove", + {"remove_hdd_data": False, "remove_backups": True}) + say(f" [X] remove (keeping drive data) -> {code} {str(d)[:160]}") + time.sleep(5) + st = stack(name) + left = guest(f"ls -d /opt/docker/stacks/{name} 2>/dev/null; " + f"docker ps -a --filter label=com.docker.compose.project={name} --format '{{{{.Names}}}}'") + say(f" [X] after remove: deployed={st.get('deployed')} leftovers={left.strip()!r}") + return code + + +def app_env(name, key): + """Read one deploy value the CUSTOMER was given (e.g. the generated admin password) from the + app's own `app.yaml`. This is not seeding — it is how the household logs in; the controller + shows them the same value. Data still goes in through the app's own front door.""" + out = guest(f"grep -E '^\\s*{key}:' /opt/docker/stacks/{name}/app.yaml 2>/dev/null | head -1") + if ":" in out: + return out.split(":", 1)[1].strip().strip('"').strip("'") + return "" + + +def snapshots(name): + """The restorable copies the backups page offers for this app.""" + code, d = ctl("GET", f"/api/backup/snapshots?stack={name}") + data = d.get("data") if isinstance(d, dict) else None + if isinstance(data, dict): + for k in ("snapshots", "items", "restore_points"): + if isinstance(data.get(k), list): + return data[k] + return data if isinstance(data, list) else [] + + +def restore(name, snapshot_id=None, wait_s=1200): + """The household's own way out: the „Visszaállítás a mentésből" button on the backups page. + + A FORM post, not an API call — `POST /backup/restore` with `_csrf`, `stack_name`, + `snapshot_id` — because that is the button the sentence tells them to press. + """ + snaps = snapshots(name) + if snapshot_id is None: + if not snaps: + say(f" [R] no restorable copy offered for {name}") + return {"ok": False, "why": "no snapshot offered", "snapshots": snaps} + first = snaps[0] + snapshot_id = first.get("id") or first.get("snapshot_id") or first.get("short_id") + say(f" [R] restoring {name} from snapshot {snapshot_id!r} (of {len(snaps)} offered)") + sess = open(f"{SC}/sess{os.getpid()}.txt").read().strip() + csrf = open(f"{SC}/csrf{os.getpid()}.txt").read().strip() + r = sh(["curl", "-sk", "-D", "-", "-o", "/dev/null", "-H", HOSTHDR, "-H", f"Cookie: {sess}", + "-X", "POST", + "--data-urlencode", f"_csrf={csrf}", + "--data-urlencode", f"stack_name={name}", + "--data-urlencode", f"snapshot_id={snapshot_id}", + f"{BASE}/backup/restore"], timeout=180) + head = (r.stdout or "").split("\n")[0].strip() + loc = [l for l in (r.stdout or "").split("\n") if l.lower().startswith("location:")] + say(f" [R] POST /backup/restore -> {head} {loc[:1]}") + t0 = time.time() + last = None + while time.time() - t0 < wait_s: + code, d = ctl("GET", "/api/backup/restore-status") + dd = d.get("data") or {} + cur = (dd.get("running"), dd.get("phase") or dd.get("state"), dd.get("message")) + if cur != last: + say(f" +{round(time.time()-t0,1):>6.1f}s restore {cur}") + last = cur + if not dd.get("running", False) and time.time() - t0 > 5: + break + time.sleep(2) + st = stack(name) + say(f" [R] after restore: state={st.get('state')} hold={st.get('hold_reason')!r} " + f"phase={st.get('update_phase')}") + return {"ok": True, "snapshot_id": snapshot_id, "snapshots": snaps, + "http": head, "location": loc[:1], "seconds": round(time.time() - t0, 1), + "state_after": st.get("state"), "hold_after": st.get("hold_reason"), + "observables_after": observables(name)} diff --git a/documentation/backlog/OPEN-ITEMS.md b/documentation/backlog/OPEN-ITEMS.md index 7fb5586c..b51dc8a5 100644 --- a/documentation/backlog/OPEN-ITEMS.md +++ b/documentation/backlog/OPEN-ITEMS.md @@ -664,7 +664,7 @@ class (an image `VOLUME` at an unmounted path) is still live — `immich-server` | **R-440** | **[P2-MEDIUM] 23 catalog image pins float, so an update is not reproducible.** `compose pull` on a moving tag fetches whatever upstream published that day. **MEASURED 2026-09-01 over `app-catalog-felhom.eu` @ `29edad9c5bf4`: 79 `image:` lines across 53 apps, 66 distinct; 23 of those lines carry a tag with no patch version.** `postgres:16-alpine` (8 apps), `redis:7-alpine` (6), `mariadb:11.6` (2), plus one each of `postgres:15-alpine`, `postgis/postgis:16-3.5-alpine`, `mariadb:11.4`, `mariadb:12.3`, `ghcr.io/claperco/claper:2.5`, `ghcr.io/thomiceli/opengist:1.13`, `wger/server:2.6`. **A 24th is arguable and is recorded rather than rounded away:** `ghcr.io/immich-app/postgres:16-vectorchord0.4.3-pgvectors0.2.0` pins both extensions exactly but leaves the PostgreSQL patch floating. A customer pressing Frissites can therefore swap their DATABASE ENGINE build with no catalog change and no record; two boxes updated on two days end up different. **Severity MEDIUM on its own; it becomes BLOCKING the moment a pre-update copy exists**, because "what did we upgrade from and to" must be recordable and today it is not — which is also why R-440 must be read next to the digest discipline in Rule 10 of the spike. **MEASURED LIVE 2026-09-01 — the floating pins have ALREADY moved, with a passing control.** Running digests on demo-hp compared against what the registry serves for the same tag today: **`mariadb:11.4` MOVED** (`sha256:4f1d8d20...` -> `sha256:611a2fcc...`) and **`mariadb:12.3` MOVED** (`sha256:a02fe89c...` -> `sha256:dd9b303a...`), while `postgres:16-alpine`, `redis:7-alpine`, `mariadb:11.6` and `opengist:1.13` were SAME — **and both fully-pinned CONTROLS (`rommapp/romm:5.0.0`, `privatebin/pdo:2.0.5`) were SAME.** So on a box with ZERO visible drift by tag, pressing Frissites today silently swaps the DATABASE ENGINE build under `romm` and `bookstack`, with no catalog change and no record. **Compounding fact found while reading:** the recovery unit records `ImagePins` but the manifest comment says *"image NOT stored - re-pulled on restore"*, so a RESTORE of a floating-pinned app also re-pulls whatever is current — the same non-reproducibility on the recovery path. **HALF OF THE ANSWER SHIPPED 2026-09-02 (controller v0.233.0, slice 1): `app.yaml.installed_images` now records, per compose SERVICE, the reference AND the repo digest each container was actually created from — so "what did we upgrade FROM" is answerable on any box that has taken one lifecycle action since the upgrade.** What is still missing is the other half: comparing that digest against what the registry serves for the same tag TODAY, which needs a network call the render path deliberately does not make (see R-446). **The row therefore stays OPEN and its rank is unchanged** — recording a digest does not make a floating pin reproducible; it makes the drift measurable after the fact. `audits/SPIKE-app-update-2026-09-01.md` **— NIGHT 2026-09-23:** every image moved tonight (and the 21 backfilled moves) carries its resolved digest in the catalog's test record; the floating pins themselves still float until the box pulls by digest (`09` §6.4 part 6). **-- MEASURED 2026-09-30 on demo-hp 9201: a fresh install or a guarded Update renders `name:tag@sha256` from the ladder (`digest.go` `RenderWithLadderDigests`) — 9 of 20 services there run such a definition (adventurelog, docmost, paperless-ngx). The other 11 run TAG-ONLY definitions: bookstack, kimai, opengist, privatebin, romm were installed before v0.269.0 and not updated since, and `CarryDigests` (`digest.go:141`) keeps only a digest the running definition already has; bentopdf and calibre-web have no ladder at all. So a re-pull of those (a restore) takes whatever the tag serves that day. `audits/pg-last-six-2026-09-30/E/E2-digests-demo-hp.txt`.** **-- 2026-09-30 (evening):** calibre-web now has a ladder entry (`53a4a1d`), so a new install or a guarded Update renders its tested digest; bentopdf still has none. `audits/more-night-apps-2026-09-30/` | **NARROWED 2026-09-30 — floating only for (1) apps with no ladder entry and (2) apps installed before v0.269.0 until their next guarded Update; owner: CC.** | | **R-444** | **[P3-LOW] Nothing runs `pct fstrim` on the fleet, and demo-hp's thin pool was carrying ~23.8 GB of blocks the guest had already freed.** MEASURED 2026-09-01 during this spike's teardown: the run itself added ~1.05 GiB that `local-lvm` did not reclaim on delete (68.97% -> 70.91%); `fstrim` INSIDE the unprivileged container is refused (`FITRIM ioctl failed: Operation not permitted`, all three mounts); `pct fstrim 9201` from the PVE host then trimmed **30.2 GiB + 57 GiB** and took `local-lvm` to **26.78%** — **23.8 GB BELOW this run's own starting point**, i.e. the surplus was long-standing, not ours. **Why it is not merely housekeeping:** a thin pool that only ever grows can reach 100% from DELETED data alone, and a full thin pool takes every guest on the host read-only. demo-hp had 16.4 GB free before the trim. **Not urgent, and the row says so** — but the appliance has no periodic trim and no operator surface reports the gap between guest-free and pool-used. Owner: **CC.** `audits/SPIKE-app-update-2026-09-01.md` | **OPEN — rank P3-LOW; owner: CC** | | **R-445** | **[P3-LOW] Hub app telemetry survives the app's removal, so a 15-minute throwaway now sets a FLEET-WIDE memory recommendation.** MEASURED 2026-09-01: this spike's Phase 6 Nextcloud existed for ~15 minutes on demo-hp, spent part of it crash-looping, and was then removed with all volumes. The hub's `/apps/nextcloud` page still reports `Deployments`, `Avg Memory 208 MB`, `P95 Memory 280 MB` and **`Suggested Limit (P95x1.2) = 352 MB`**, plus three MariaDB `io_uring` rows under Known Issues attributed to demo-hp. **The suggested limit is an operator-facing recommendation derived from a sample that no longer exists anywhere** — and Nextcloud is a real catalog app whose limit someone may act on. **RETAINED DELIBERATELY BY THIS RUN, NOT CLEARED, and the reason is part of the row:** the hub offers `POST /apps/nextcloud/reset-telemetry` whose own confirm reads *"Delete all telemetry data for nextcloud? This cannot be undone."* — an irreversible write on the operator's surface, and the operator authorised Phase 6, not this. **The one-line command is recorded in the audit doc so it is a decision, not a task.** The general question is the row: should telemetry for an app with zero live deployments age out, or be excluded from the suggestion? Owner: **VIKTOR rules, CC implements.** `audits/SPIKE-app-update-2026-09-01.md` | **OPEN — rank P3-LOW; owner: VIKTOR rules, CC implements** | -| **R-446** | **[P2-MEDIUM] „Naprakész" can be FALSE, and the badge that says it cannot tell.** Slice 2 (controller v0.233.0, 2026-09-02) compares the RECORDED image reference per compose service against the reference the current template pins, and **queries no registry** — deliberately: a customer's box must not depend on reaching eight upstream registries to render a page (`felhom-controller/controller/internal/web/updatebadge.go`, `compareInstalledToTemplate`). **For the 23 floating pins that comparison is blind by construction:** `postgres:16-alpine`, `mariadb:11.6` and 21 others can carry an identical reference over an image that has moved. **MEASURED, not theorised — spike §5 found `mariadb:11.4` and `mariadb:12.3` had BOTH already moved upstream while two fully-pinned CONTROLS held.** So `romm` and `bookstack` on demo-hp would read „Naprakész" over a database engine build that is not the one the catalog now resolves to. **This is a KNOWN LIMITATION OF A SHIPPED FEATURE, filed the same session rather than left implicit**, and it is stated in the same words in `architecture/09-update-architecture.md` §8.1 and in the controller's `README.md`. The close is a digest comparison against the registry, which needs a network call, a cache and a failure posture — it is not a one-liner and it is not slice 2's job. **Depends on R-440**, whose fix (stop floating) would remove the problem instead of measuring it — take that route first if it is available. `architecture/09-update-architecture.md` **MEASURED 2026-09-21, and the blind spot is not one or two pins.** `audits/UPDATE-ARC-STATE-2026-09-21.md` §3.3: the catalog carries **10 floating pins of 66** (recounted — the old "23" was stale), and **6 of the 7 measurable engine pins have been repushed upstream since the catalog set them** — `postgres:16-alpine` (8 apps), `postgres:15-alpine`, `redis:7-alpine` (6 apps), `mariadb:11.4`, `mariadb:12.3`, `postgis:16-3.5-alpine`; only `mariadb:11.6` has not. The 8th (immich's own ghcr build) is UNMEASURED — ghcr exposes no anonymous last-modified timestamp. **So on demo-hp today four apps read „Naprakész" over a database engine image that has demonstrably moved.** The fix does NOT need the box to query a registry: the catalog can record each pin's digest at push time (`check-image-resolvable.py` already resolves it) and the box compares digests. Put to the operator as `09` §3b **Q6**, recommended YES — the cheapest real improvement on the arc's list. **— UPDATE NIGHT 2026-09-21:** **MEASURED ON A BOX 2026-09-21 (update night, leg B8), and it REFINES the row in two ways rather than merely confirming it.** §8.1's numbers came from a registry sweep on DooPlex; this is the same question asked of a customer-shaped box, where the badge actually renders. On guest 9202, `docmost`'s two floating pins were read as `installed_images` records them and compared against the upstream digests measured the same night: `postgres:16-alpine` → **`sha256:721873c34ceb9…` on the box and `sha256:721873c34ceb9…` upstream**, and `redis:7-alpine` → **`sha256:858f009f9709c…` both sides**. **Identical. So the badge „Naprakész" is TRUE for this box**, and the app reads correctly. **(1) The defect's size is set by INSTALL AGE, not by the catalog.** A floating pin is wrong only for a box that pulled BEFORE the tag moved; a box deployed after the repush holds the current image and its badge is right. R-446's "six repushed pins" measured the tag against the date the CATALOG set it, which is the right measure for *the catalog* and not for *a box*. **(2) The producer Q6 needs ALREADY EXISTS on the box.** `installed_images` records a real `digest` per service (`installed.go` §7.1) — the box knows exactly what it is running. What it cannot do is COMPARE, because the catalog carries no digest to compare against. That is Q6's proposal, and this is a concrete confirmation that only the catalog half is missing. Evidence: `audits/update-night-2026-09-21/23-B8-floating-pin.txt`. **-- RULED 2026-09-23 (`09` §3 decision 17):** YES — the catalog records the image digest of every pin at push time; the box compares against it and, where the catalog carries one, pulls **that exact image**, which makes a floating tag reproducible, not only the badge honest. *Pull-by-digest while the definition names a tag is a claim to verify in the build, not a ruling on mechanism.* **-- 2026-09-23:** pull-by-digest MEASURED on 9202 — Docker and Compose both pull and run `redis:7-alpine@sha256:858f…` and refuse a digest that does not exist. **Build trap, read from source:** `splitImageRef` returns "unorderable" for any ref containing `@` (`stacks/updateorder.go:134`), so a digest-carrying pin must have its digest split off before ordering or every such app reads Unknown. `09` §6.4 part 6. **— NIGHT 2026-09-23:** the CATALOG half of the close shipped: every ladder entry records the digest the registry served for each `to` ref (`scripts/image_digest.py`), and the move gate refuses a digest the registry no longer serves. The box does not compare it yet (`09` §6.4 part 6, box half). **-- MEASURED 2026-09-30 on demo-hp 9201 (controller 0.283.1): the box half of `09` §6.4 part 6 is live — `stacks/updateorder.go:86` `digestBehind` compares the ladder's tested digest (`catalog_digests`) with `app.yaml installed_images[svc].digest`. All 18 services of the 8 ladder apps there carry both, and all 18 are equal, so their „Naprakész" is true. STILL BLIND by construction (`updateorder.go:96`): an app with NO ladder entry — the catalog carries no digest to compare (bentopdf, calibre-web on demo-hp; 21 of 53 templates after 2026-09-30). `audits/pg-last-six-2026-09-30/E/E2-digests-demo-hp.txt`.** **-- 2026-09-30 (evening): 15 of 53 templates now carry no ladder** (calibre-web, gitea, wger, crafty-controller, uptime-kuma, zipline got their first proven step) — the badge is blind by construction only for those 15. **Separately, R-740:** for a floating tag the catalog never records a NEWER tested digest, so the badge's digest comparison can read „Naprakész" while upstream has moved. `audits/more-night-apps-2026-09-30/` | **NARROWED 2026-09-30 — blind only for apps with no ladder entry (15 of 53 by the evening; see also R-740); closes as each gets its first proven step (R-462). Owner: CC.** | +| **R-446** | **[P2-MEDIUM] „Naprakész" can be FALSE, and the badge that says it cannot tell.** Slice 2 (controller v0.233.0, 2026-09-02) compares the RECORDED image reference per compose service against the reference the current template pins, and **queries no registry** — deliberately: a customer's box must not depend on reaching eight upstream registries to render a page (`felhom-controller/controller/internal/web/updatebadge.go`, `compareInstalledToTemplate`). **For the 23 floating pins that comparison is blind by construction:** `postgres:16-alpine`, `mariadb:11.6` and 21 others can carry an identical reference over an image that has moved. **MEASURED, not theorised — spike §5 found `mariadb:11.4` and `mariadb:12.3` had BOTH already moved upstream while two fully-pinned CONTROLS held.** So `romm` and `bookstack` on demo-hp would read „Naprakész" over a database engine build that is not the one the catalog now resolves to. **This is a KNOWN LIMITATION OF A SHIPPED FEATURE, filed the same session rather than left implicit**, and it is stated in the same words in `architecture/09-update-architecture.md` §8.1 and in the controller's `README.md`. The close is a digest comparison against the registry, which needs a network call, a cache and a failure posture — it is not a one-liner and it is not slice 2's job. **Depends on R-440**, whose fix (stop floating) would remove the problem instead of measuring it — take that route first if it is available. `architecture/09-update-architecture.md` **MEASURED 2026-09-21, and the blind spot is not one or two pins.** `audits/UPDATE-ARC-STATE-2026-09-21.md` §3.3: the catalog carries **10 floating pins of 66** (recounted — the old "23" was stale), and **6 of the 7 measurable engine pins have been repushed upstream since the catalog set them** — `postgres:16-alpine` (8 apps), `postgres:15-alpine`, `redis:7-alpine` (6 apps), `mariadb:11.4`, `mariadb:12.3`, `postgis:16-3.5-alpine`; only `mariadb:11.6` has not. The 8th (immich's own ghcr build) is UNMEASURED — ghcr exposes no anonymous last-modified timestamp. **So on demo-hp today four apps read „Naprakész" over a database engine image that has demonstrably moved.** The fix does NOT need the box to query a registry: the catalog can record each pin's digest at push time (`check-image-resolvable.py` already resolves it) and the box compares digests. Put to the operator as `09` §3b **Q6**, recommended YES — the cheapest real improvement on the arc's list. **— UPDATE NIGHT 2026-09-21:** **MEASURED ON A BOX 2026-09-21 (update night, leg B8), and it REFINES the row in two ways rather than merely confirming it.** §8.1's numbers came from a registry sweep on DooPlex; this is the same question asked of a customer-shaped box, where the badge actually renders. On guest 9202, `docmost`'s two floating pins were read as `installed_images` records them and compared against the upstream digests measured the same night: `postgres:16-alpine` → **`sha256:721873c34ceb9…` on the box and `sha256:721873c34ceb9…` upstream**, and `redis:7-alpine` → **`sha256:858f009f9709c…` both sides**. **Identical. So the badge „Naprakész" is TRUE for this box**, and the app reads correctly. **(1) The defect's size is set by INSTALL AGE, not by the catalog.** A floating pin is wrong only for a box that pulled BEFORE the tag moved; a box deployed after the repush holds the current image and its badge is right. R-446's "six repushed pins" measured the tag against the date the CATALOG set it, which is the right measure for *the catalog* and not for *a box*. **(2) The producer Q6 needs ALREADY EXISTS on the box.** `installed_images` records a real `digest` per service (`installed.go` §7.1) — the box knows exactly what it is running. What it cannot do is COMPARE, because the catalog carries no digest to compare against. That is Q6's proposal, and this is a concrete confirmation that only the catalog half is missing. Evidence: `audits/update-night-2026-09-21/23-B8-floating-pin.txt`. **-- RULED 2026-09-23 (`09` §3 decision 17):** YES — the catalog records the image digest of every pin at push time; the box compares against it and, where the catalog carries one, pulls **that exact image**, which makes a floating tag reproducible, not only the badge honest. *Pull-by-digest while the definition names a tag is a claim to verify in the build, not a ruling on mechanism.* **-- 2026-09-23:** pull-by-digest MEASURED on 9202 — Docker and Compose both pull and run `redis:7-alpine@sha256:858f…` and refuse a digest that does not exist. **Build trap, read from source:** `splitImageRef` returns "unorderable" for any ref containing `@` (`stacks/updateorder.go:134`), so a digest-carrying pin must have its digest split off before ordering or every such app reads Unknown. `09` §6.4 part 6. **— NIGHT 2026-09-23:** the CATALOG half of the close shipped: every ladder entry records the digest the registry served for each `to` ref (`scripts/image_digest.py`), and the move gate refuses a digest the registry no longer serves. The box does not compare it yet (`09` §6.4 part 6, box half). **-- MEASURED 2026-09-30 on demo-hp 9201 (controller 0.283.1): the box half of `09` §6.4 part 6 is live — `stacks/updateorder.go:86` `digestBehind` compares the ladder's tested digest (`catalog_digests`) with `app.yaml installed_images[svc].digest`. All 18 services of the 8 ladder apps there carry both, and all 18 are equal, so their „Naprakész" is true. STILL BLIND by construction (`updateorder.go:96`): an app with NO ladder entry — the catalog carries no digest to compare (bentopdf, calibre-web on demo-hp; 21 of 53 templates after 2026-09-30). `audits/pg-last-six-2026-09-30/E/E2-digests-demo-hp.txt`.** **-- 2026-09-30 (evening): 15 of 53 templates now carry no ladder** (calibre-web, gitea, wger, crafty-controller, uptime-kuma, zipline got their first proven step) — the badge is blind by construction only for those 15. **Separately, R-740:** for a floating tag the catalog never records a NEWER tested digest, so the badge's digest comparison can read „Naprakész" while upstream has moved. `audits/more-night-apps-2026-09-30/` **-- 2026-09-30 late (decision 52):** a floating tag's badge can now go honestly behind by digest — the catalog records a re-test at the new digest (`retest-floating.py`), and the box's `digestBehind` reads it (proven on 9202, badge „Frissítés elérhető” → „Naprakész”). | **NARROWED 2026-09-30 — blind only for apps with no ladder entry (15 of 53 by the evening; see also R-740); closes as each gets its first proven step (R-462). Owner: CC.** | | **R-450** | **[P2-MEDIUM] UPDATE ARC SLICE 6 — a version sequence: automatic WITHIN a major, never ACROSS one, and an engine change gets its OWN edge.** The first half is an operator ruling of 2026-09-02 and its justification is R-449's measurement: a cross-major jump can be refused by the app itself and cannot be undone. **The second half is a rule recorded now, while it is cheap:** an engine change must never be bundled with an app version bump. `bookstack`'s `0b73e5e` moved the application 25.02.2 → 26.05.2 **and** MariaDB 11.6 → 12.3 in one commit — **two migrations behind one edge**, and an unreadable failure when it breaks. Needs a catalog-side convention and, eventually, a gate. `architecture/09-update-architecture.md` §6 **HALF SHIPPED 2026-09-21 (catalog `5ff36d098cbc`): the second half — an engine change gets its OWN edge — is now ENFORCED** by `check-engine-major.py`, which refuses a commit moving a MariaDB major together with any other image move in that template, naming what it was bundled with. The FIRST half (automatic within a major) is Slice 6 and needs four operator answers — `09` §3b **Q1–Q4**, with the shape it would take in `09` §6.2. **The urgency is now measured:** 46 of the catalog's 58 exact pins are behind upstream and **39 of those are within a major** — the population the 2026-09-02 ruling already says may move without a human. **-- RULED 2026-09-23 (operator, `09` §3 decisions 11–15):** Q1–Q4 answered. The update is a leg of the backup chain after off-site and before the full-system backup (11); automatic with a per-box switch ON by default (12); **the TEST decides, not the tag** — the box applies every step the catalog holds because the catalog holds only tested steps, and `CompareImageRefs` moves to the catalog gate (13, REPLACES "never across a major"); a box behind climbs **one tested step at a time** (14); **the box UNDOES a failed update itself** — old definition + the pre-pin safety dump + health check again, HOLD only if the undo fails (15, REPLACES §6.1's no-auto-undo). The undo and the ladder were SPIKED the same day before any build (`audits/update-rulings-2026-09-23/`); build order and costs in `09` §6.4. **-- SPIKED 2026-09-23 (`audits/update-rulings-2026-09-23/`):** the undo works by hand on three real migrating edges and needs eight product additions (R-637..R-642); **the ladder is measured absent** — one press on a box two steps behind jumped vikunja 2.3.0 → 2.5.0 in 9.5 s and 2.4.0 never ran, and the box cannot see intermediate steps at all because its catalog clone is `--depth 1` (`sync.go:283`/`:300`, one commit visible on both demo guests). The ladder's recommended format is an `update_ladder:` list in `.felhom.yml` with each intermediate step's own definition, NOT the git history (romm's image-moving commit is the definition that OOM-looped). The chain's update leg has ≤15 min as ruled (R-643). Build order `09` §6.4. **— NIGHT 2026-09-23: `09` §6.4 part 4 SHIPPED** (catalog `6db08a5`): the test record `update_ladder:` + two gates + the only writer + the 21-move backfill; 12 more steps published with records. Parts 5 (the box climbs), 6-box-half and 7 remain; the romm press on demo-hp showed today's jump live — 5.3.0 → 5.3.1 AND mariadb 11.4 → 11.8 in one press (both tested steps; `done`). **— 2026-09-24: `09` §6.4 PART 5 SHIPPED** (controller v0.268.0 `206b035`, catalog `5ed599c`): one press = one tested step, each step's own definition at `templates//steps/.yml`, proven live on 9202 (romm 5.3.0/11.4 → 5.3.1/11.4 → 5.3.1/11.8 in two presses, `audits/ladder-2026-09-24/partD/`). Left in this row: part 6's box half, part 7 (the automatic leg), part 10 (PostgreSQL majors). **— 2026-09-25 night: part 6's box half shipped in v0.269.0/v0.269.1; PART 7 SHIPPED as controller v0.271.0** (the automatic leg, proven over six simulated nights on 9202 and watched through the demo boxes' first real night, `audits/DRILL-night-2026-09-25.md`). **Left: part 10 only (PostgreSQL majors, R-463).** **-- 2026-09-25 (evening): part 10 SHIPPED for docmost** (controller v0.273.0, catalog `afd3a60`, decisions 35–39; `audits/night-2026-09-26/`). What stays open: each of the other ten PostgreSQL apps needs its own two-venue proof before the gate lets its major move. | **NARROWED — the other ten PostgreSQL apps; owner: CC** | | **R-451** | **[P3-LOW] UPDATE ARC SLICE 7 — a fleet sweep: the operator can SEE, and MOVE, how far behind every box is.** Slices 1 and 2 make one box's state visible on that box's own pages. The operator has no fleet view, and **it is not derivable from what is already reported: the hub's report payload carries container name, state, CPU and memory, and NO image field at all** (spike §5, which is why Peti's box could only be recorded UNKNOWN). So this is a hub-side change as well as a controller one. Rank LOW today because the fleet is two enrolled boxes; it rises with the fleet. `architecture/09-update-architecture.md` §6, §8.4 **BOTH SIDES VERIFIED 2026-09-21, and it is cheaper than this row implies.** The controller's payload carries no image (`internal/report/types.go` L98–103) and the hub's `Store.SaveReport` (`hub/internal/store/store.go:965`) denormalises only container **counts** — but **the hub stores the raw report JSON whole**, so a new controller field lands there the day it is sent. What is missing is the denormalisation and the page, not the transport. Shape in `09` §6.2–6.3; the payload question is `09` §3b **Q7**. **-- RULED 2026-09-23 (`09` §3 decision 18):** the report carries, per compose service (database included), the installed reference, the catalog reference and the badge state. **Built later, when the fleet grows** — Q7's recommendation, confirmed. | **RULED — build deferred until the fleet grows; owner: CC** | | **R-454** | **[P3-LOW] Five `internal/web` test files have been `gofmt`-unclean for an unknown length of time, and nothing notices.** MEASURED 2026-09-02: `gofmt -l controller/internal/web/` reports `backups_split_test.go`, `claim_code_naming_test.go`, `disk_health_test.go`, `r400_debug_routes_test.go`, `recovery_test.go` — at the **baseline** commit `960d29b0612c`, i.e. not introduced by v0.233.0 (both files added that day are clean). **`go vet` does not check formatting and `controller_gates.py` has no formatting gate**, so the only thing that would ever surface this is someone running `gofmt -l` by hand, which is how it was found. **Not reformatted in the same session, deliberately** — the minimal-changes rule, and a five-file whitespace commit inside a feature release makes that release's diff unreadable. **Small, and the cost of NOT having the instrument is the row:** the count can only grow, and every future `gofmt -l` run produces noise that hides a real one. Fix is two lines: a `gofmt -l` gate in `controller_gates.py` plus one formatting commit, in that order (the gate first, so the commit is provably complete). Owner: **CC.** | **READY — rank P3-LOW; owner: CC** | @@ -810,7 +810,7 @@ class (an image `VOLUME` at an unmounted path) is still live — `immich-server` | **R-688** | **[P3-LOW] The customer delete says it removes the tunnel and zone, but no leg of it calls Cloudflare.** The dialog's acknowledgement reads "the customer will be RESET — offsite repo DESTROYED, PBS revoked, tunnel/zone removed" (`hub/internal/web/customer_delete.go` `deleteCascadeAcks`), while `commitCustomerReset` has legs for Hetzner, PBS, claim, descriptor and DB only. Seen 2026-09-25 retiring `peti-felhom`, whose config carried a Cloudflare tunnel token and API token (`sajatfelhom.hu`): the tokens went with the record; any tunnel or DNS record on Cloudflare's side was neither listed nor removed. **Fix direction:** either a Cloudflare leg (tunnel + DNS by the customer's ids), or the dialog stops promising it and lists what to remove by hand. `audits/RETIRE-peti-2026-09-25.md` **-- HALF DONE 2026-09-25 (hub v0.125.0):** the dialog no longer promises a Cloudflare removal; the preview lists what the operator removes by hand, by domain (the tunnel, the DNS records), never the token — proven live on the hub (`audits/night-2026-09-26/F/`). The Cloudflare leg itself is NOT built. | **NARROWED — the Cloudflare leg only; owner: operator (decide if it is wanted) / CC (build)** | | **R-691** | **[P3-LOW] Kept data (09 §3 decision 36): two gaps of the first build.** (1) **The read-only file-browser view cannot open a folder another user owns with mode 0770** — nextcloud's `appdata/nextcloud` is `www-data` `drwxrwx---` (measured on 9202 2026-09-25), FileBrowser runs as uid 1000, so „Megőrzött adatok" shows the folder and not its files; the files are still listed, sized, loadable and deletable. Fix direction needs a decision (a read-only ACL, or a helper that lists as root) — not a chmod of the household's data. (2) **„Use my kept data" / Load looks only at the own unit (Tier 1) and the second-drive mirror (Tier 2)**; an app whose only database copy is off-site gets "no backup". Controller `43e99d1`. `audits/night-2026-09-26/E/` **-- 2026-09-25 live proof:** (1) confirmed on 9202 — the view mounts nextcloud's kept folders `:ro` but its files are `www-data` 0770. Also seen: the source's name „Megőrzött adatok" is Hungarian on an English box (the file browser's config holds one name). **-- 2026-09-27 (controller v0.275.0): (1) FIXED: the view joins the kept folder's OWNING GROUP when it is group-readable (never root's, never its own), binds stay `:ro`, nothing on disk changes (CC-unattended decision, `07` §6.5); the source's name follows a language switch (the switch re-syncs the file browser). Red-proofed, `audits/version-travel-2026-09-26/D3/`. NOT live-proven with a real nextcloud kept folder. STILL OPEN: (2), the Use/Load choice does not look at the off-site copy.** **-- 2026-09-27 (second session): (2) NOT built on purpose** — it composes the unit-only off-site download (`RestoreOffboxScratch(full=false)`) with the unit restore into a new restore path on household data, and no box CC may touch has an off-site target to prove it on (9202 has none; 9201 on both demo hosts is fenced). Needs: a Tier-0 guest with an off-site target, or an operator word to use one.** **-- 2026-09-28 (controller v0.277.0): (2) BUILT** — `KeptBestCopy` offers the off-site copy when it is newer than every local copy or the only one; the page names the copy and its date; `LoadKeptOffsite` downloads the unit alone, refuses a unit of another drive, with no data, or with no recorded data version (`07` §6.5/§6.6), then restores. Red-proofed (`audits/kept-offsite-2026-09-28/redproofs/`). Tier 1 regression live on 9202 (the choice named „saját mentés, 2026-09-28 10:06”, seed + file back). Floor 0.277.0, both demo boxes. **STILL OPEN: the live off-site proof** — a throwaway nextcloud on demo-hp 9201 joined the off-site copy 2026-09-28 10:15; its first snapshot runs the night of 09-28/29 (Part E (b)). Tier 2 not runnable live (9202 has one drive). **-- 2026-09-28 afternoon: LIVE-PROVEN on demo-hp 9201 (controller 0.278.0), endpoint level.** A throwaway nextcloud, seeded through its own front door, joined the off-site copy; the off-site run-now pushed snapshot `6cb379a8`. (a) The full off-site restore (prepare → download → reconstitute): 3 volumes + the database replayed, the seed read back, a marker user written after the snapshot read ABSENT, same versions. (b) Remove keeping the data + deleting the local copies → reinstall: the choice and the kept list both named "távoli mentés, 2026-09-28 15:40" / "the off-site copy, 2026-09-28 15:40"; "use my kept data" downloaded the unit alone and loaded 3/3 volumes + 1/1 database in 55 s; the seed and the kept files read back. App removed with its data; demo-hp's app list equals the list before. `audits/kept-offsite-2026-09-28/E/`. | **CLOSED — controller v0.277.0, live 2026-09-28** | | **R-693** | **[P3-LOW] The memory watch marks a Node app `memory_tight` at any limit — its heap sizes itself from the limit.** Measured 2026-09-25 on the bench (docmost 0.96.0, harness v4): the app's own memory (`anon`) peaked at **349 MB of 384 MB (90.9 %)**, then, with the limit raised to 512 MB, at **431 MB of 512 MB (80.4 %)** — 0 OOM kills and 0 restarts in both 10-minute watches (~12 000 requests each). So the mark (decision 22's "does not fit the memory") fires for an app that fits, and the gate's remedy (raise the limit) cannot clear it. docmost moved with the limit raised to 512 MB (decision 39). **Needs:** a basis that tells growth-to-fill from pressure (e.g. kills/restarts plus a GC-pressure signal, or a second watch at a higher limit showing the peak scales), or a per-app `memory_scales_with_limit` fact. `audits/night-2026-09-26/C/bench-run1/`, `…/bench-run2/` | **OPEN — P3; owner: CC** | -| **R-698** | **[P3-LOW] A backup stores the image's NAME, not the image — a restore of a version its maker has deleted cannot start.** `RecoveryManifest.image_pins` ("image NOT stored — re-pulled on restore"); since controller v0.275.0 each data file also records its running `ref@digest`, and a restore brings the data back AT ITS OWN VERSION (`07` §6.6) — so a restore asks for exactly the old image. **Measured 2026-09-26** (`audits/version-travel-2026-09-26/A7/`, registry HEADs, no pulls): the catalog's 42 ladder `ref@digest` pairs all resolve (200); an invented digest answers 404 on Docker Hub and ghcr.io (negative control). Not measured: the digests recorded on boxes (older than any ladder entry), how often makers delete versions, the catalog's 66 digest-less compose lines. **Options (decide nothing yet):** (a) keep — a restore of a deleted version fails at the pull and the household uses the next copy or a newer version; (b) mirror every INSTALLED image into the DooPlex registry, restore falls back to it — storage + bandwidth on DooPlex, a new part on the recovery path; (c) mirror only ladder-named versions — bounded, misses pre-ladder boxes; (d) `docker save` into the unit — hundreds of MB per app per copy on every tier. | **OPEN — P3; owner: operator (a decision), CC measures** | +| **R-698** | **[P3-LOW] A backup stores the image's NAME, not the image — a restore of a version its maker has deleted cannot start.** `RecoveryManifest.image_pins` ("image NOT stored — re-pulled on restore"); since controller v0.275.0 each data file also records its running `ref@digest`, and a restore brings the data back AT ITS OWN VERSION (`07` §6.6) — so a restore asks for exactly the old image. **Measured 2026-09-26** (`audits/version-travel-2026-09-26/A7/`, registry HEADs, no pulls): the catalog's 42 ladder `ref@digest` pairs all resolve (200); an invented digest answers 404 on Docker Hub and ghcr.io (negative control). Not measured: the digests recorded on boxes (older than any ladder entry), how often makers delete versions, the catalog's 66 digest-less compose lines. **Options (decide nothing yet):** (a) keep — a restore of a deleted version fails at the pull and the household uses the next copy or a newer version; (b) mirror every INSTALLED image into the DooPlex registry, restore falls back to it — storage + bandwidth on DooPlex, a new part on the recovery path; (c) mirror only ladder-named versions — bounded, misses pre-ladder boxes; (d) `docker save` into the unit — hundreds of MB per app per copy on every tier. **-- 2026-09-30 late (decision 53):** a box now keeps only an app's running and previous image; a restore to an older version re-pulls it — as every restore already did. The limit above is unchanged. | **OPEN — P3; owner: operator (a decision), CC measures** | | **R-700** | **[P2] A drive move unpinned the app — its next start took the catalog's newest version, past the ladder.** Found 2026-09-27 reading the code for R-697 (not seen on a box): `doFlipRedeploy` (the per-app and whole-drive move) persisted through the restore's fresh `app.yaml` write, which drops `pinned_images`, `desired_state`, `installed_images`, the update records and the kept conversion copies. Unpinned, the catalog syncer copies the catalog's compose verbatim (`sync.renderSource`'s table) and the next `up` runs the newest version — for a PostgreSQL app past its conversion step, i.e. a new engine on an old datadir. Pin adoption repairs it only at a controller restart. **-- 2026-09-27 (controller v0.276.0): FIXED** — `persistDriveFlip` changes `HDD_PATH` and nothing else; red-proofed (`audits/records-carried-2026-09-27/redproofs/RP3`, `RP4`). **STILL OPEN: the live proof** — no Tier-0 guest has two drives (9202 has one); prove a move on a box with a second drive, reading `pinned_images` before and after and the running image after the next sync. | **WATCHING — P2; owner: CC (live proof)** | | **R-701** | **[P3-LOW] demo-hp's whole-guest restore test can never run: every 6 h it picks the right archive and the space preflight refuses it.** Read 2026-09-28 (agent 0.137.0, `audits/version-travel-2026-09-26/D1/D1-cycle-demo-hp.txt`): 20:13 and 02:13 CEST both skipped the golden file and the deleted guest 9100's archive (R-689 working), chose `felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z` (21.6 GiB), and were refused — "needs 31.0 GiB free, has 21.1 GiB" on `local-lvm` — logged `ERROR scheduled restore-test FAILED`. Same refusal first seen 2026-09-24 (R-672's delivery). So demo-hp's whole-guest tier is never proven, and the refusal is SAFE (nothing created). Not measured: whether each refusal reaches the hub or the operator as a failure. **Options (decide nothing yet):** (a) reclaim thin-pool space (`pct fstrim`, R-444) and see if 31 GiB frees; (b) restore-test into `nvme-scratch` instead of `local-lvm` — a config change on the host; (c) accept: demo-hp is a small box, record the tier as not testable there. **-- 2026-09-28 option (a) MEASURED — NOT ENOUGH:** `pct fstrim` 9201 + 9202 from the host (rc 0): `local-lvm` 62.32 % → **50.60 %**, free 20.3 → **26.6 GiB** — still under the 31 GiB the preflight needs. The pool is 53.9 GiB and guest 9201 itself holds ~26 GiB, so no trim can reach 31 GiB. The agent's own verdict after the trim: `tier=felhom-pbs due=true` (archive 2026-09-24T20:06:25Z, not proven). `nvme-scratch` has ~820 GiB free. **Left to the operator: (b) or (c).** `audits/evidence-golden-0276-2026-09-28/phaseD1-reclaim.txt` **-- 2026-09-28 14:13 CEST:** the first scheduled cycle after the trim was refused again ("needs 31.0 GiB free, has 23.9 GiB"). **Answered: each refusal DOES reach the hub** — `[WARN] host demo-hp-bb76ea restore-test FAILED …` at every host-report (every 15 min). **-- 2026-09-28 evening: CLOSED by option (b) (operator, `09` §3 decision 44).** demo-hp `restore_storage` → `nvme-scratch`; Proxmox first refused it (403 `Datastore.AllocateSpace` — the agent had no grant there, the 03 doc said so); with the operator's word the agent's `FelhomAgentStore` role was granted on `/storage/nvme-scratch` (user + token), and one restore test passed: restored + booted + verified + torn down in 8m46s, NVMe 50.3 → 74.1 GiB used at peak → back, `local-lvm` 58.46 % throughout. **The first SCHEDULED cycle (22:25) passed too:** preflight on nvme-scratch, restored, `scheduled restore-test passed` in 516 s (`C/C5-scheduled-cycle.txt`). `audits/logins-nvme-2026-09-28/C/`. | **CLOSED — option (b), 2026-09-28** | | **R-702** | **[P1-HIGH] Every claper install creates an admin `admin@claper.co` with the public password `claper`, and the app is published on the household's domain.** Measured 2026-09-28 on scratch guest 9202 (catalog `claper` template, `ghcr.io/claperco/claper:2.5` = 2.5.1): the image's own start command runs `Claper.Release.seeds`, which logs `Created default admin user: Email: admin@claper.co`; asked through claper's own CLI (`bin/claper rpc`), `get_user_by_email_and_password("admin@claper.co", "claper")` answered **true**, an unknown e-mail answered false (control). The template routes `.` through traefik and the tunnel, so any claper a household installs can be logged into by anyone who knows claper's README. Not measured: whether any box runs claper today (R-632 lists it as never deployed), whether upstream reads an env var for the seed admin. **Needs:** a decision on the fix shape — (a) the catalog passes a generated admin password (if upstream supports it), (b) the controller changes the seeded admin's password after the first start, (c) pull claper from the catalog until (a)/(b). Evidence: `audits/pg-calcom-claper-2026-09-28/box/C0-claper-default-admin.txt`. **-- 2026-09-28 evening: claper FIXED (catalog `9dc8a05`, controller v0.279.0 `after_install`)** — on a fresh install the seeded password is replaced by a generated one shown on the app page; measured on 9202: the default no longer authenticates, the generated one does, a wrong one does not, a restore keeps it. **The widened question (every app, operator ruling = decision 45) continues in R-707.** | **CLOSED — claper fixed 2026-09-28; the class → R-707** | @@ -847,13 +847,19 @@ class (an image `VOLUME` at an unmounted path) is still live — `immich-server` | **R-733** | **[P3-LOW] The test bench has NO swap and the boxes have 512 MiB — so a box proof can pass on swap where the bench fails, and nobody records whether a customer guest has swap.** MEASURED 2026-09-30 (R-732): immich's first start was OOM-killed 61–104 times on the bench (swap 0) and passed on 9202 by swapping ~108 MB; the bench given 512 MiB swap passed too. demo-hp 9201, 9202 and demo-felhom 9201 all read `swap: 512`; the golden's guest config is not recorded in its bake evidence, so a customer guest's swap is NOT measured. The harness's memory watch judges `anon` against the limit and never reads `memory.swap.current`. **Needs:** the golden's `swap` read and recorded; the box walk and the harness report `memory.swap.peak` beside `anon`; a decision whether proofs run with swap off (the stricter venue, as R-732's fix was proven). | **READY — rank P3-LOW; owner: CC (harness + golden evidence)** | | **R-734** | **[P3-LOW] The harness marks immich `files_may_change` because immich rewrites six 13-byte `.immich` folder markers at every start.** MEASURED 2026-09-30 on the bench (v3.2.2 → v3.2.4): the bind-tree hash of `appdata/immich` changed; the only changed files were `{encoded-video,library,backups,profile,thumbs,upload}/.immich`, rewritten at each start — no household file. The mark is honest by the harness's rule and the ladder writer copies it (never edited by hand), so immich's v3.2.4 night step needs a fresh WHOLE copy (decision 13); on a box without one the night leg skips it and a person presses. The 2026-09-23 immich entry did not carry it (`files_changed []`). **Needs:** a decision whether app-owned marker files are excluded from the file hash (a per-template ignore list, or a size/name rule), or the mark stays. **-- 2026-09-30 (evening):** the harness now NAMES the files behind the mark (`files_changed_detail`, catalog `5b1972b`); on immich's step `0b82…` re-proof it named exactly the six `.immich` markers again. calibre-web's step (v4.0.6 → v4.0.8) carries the mark too, and the named files are its LIBRARY DATABASE: `media/books/metadata.db`, `metadata.db-shm`, `metadata.db-wal` changed; the book file did not (bench names re-run, `A/calibre-names/`). That is household data (the template's backup class `mandatory` holds DB and books as one unit), so the mark is right there and the night leg takes that step only with a fresh whole copy. On 9202 the same step changed no file in that folder (per-file hashes before/after) — not explained. `audits/more-night-apps-2026-09-30/` | **READY — rank P3-LOW; owner: CC (harness); the rule change needs a word** | | **R-735** | **[P3-LOW] The test bench generates a `password:N:special` deploy field WITHOUT a special character, so calibre-web's own password rule refuses it and the bench cannot seed the app.** FOUND 2026-09-30 (more-night-apps brief, Part A) on the bench's second calibre-web run: the fixture ran the template's own `after_install` (`cps.py -s admin:`, the product's command) and the app answered „Password doesn't comply with password validation rules". Cause, read from source: `check-volume-persistence.py` `_gen` treats `password:24:special` as `password:24` (letters and digits); the controller's `generateValue` (`deploy.go` L1150–1164, `randomWithSpecial` L1335) adds one of `-_.!@#%+=` and at least one lower, upper and digit. The box walk was not affected (its own generator writes `Drill-`). **Instrument, not product.** **Needs:** `_gen` mirrors `randomWithSpecial`, with a test seen failing first. `audits/more-night-apps-2026-09-30/bench/apps/calibre-web/` **-- FIXED 2026-09-30 (catalog `5b1972b`):** `_gen` mirrors `randomWithSpecial`; test `test_password_special_carries_the_controllers_shape` seen failing first (length 32, no special), then 45/45 (`A/R735-red.txt`, `A/R735-green.txt`); calibre-web then proven on the bench. | **CLOSED 2026-09-30 — catalog `5b1972b`** | -| **R-736** | **[P2-MEDIUM] Removing an app or updating it never deletes the image it pulled, so a box's Docker disk fills with old images until an install or an update is refused.** MEASURED 2026-09-30 on scratch guest 9202: a wger install was refused „Docker volume below reserved buffer (4.3G free, reserve 6.8G of 68G)" — correctly; the disk held **84 images, 57 GB, 53 GB of them used by no container** (`docker system df`), left by earlier drills' installs, updates and removals through the product. Read from source: the remove runs `docker compose down --rmi local --volumes` (`felhom-controller` `stacks/delete.go:291`); `--rmi local` removes only images with no custom tag, i.e. never an image pulled by a registry tag. No image clean-up exists anywhere else in the controller (searched: `rmi`, `image prune`, `ImageRemove`). A guarded Update leaves the previous version's image too (it is also the undo's image — keeping the LAST one is deliberate; keeping every one is not decided anywhere). **A household box follows the same path more slowly:** every automatic step adds an image, and the refusal lands on the next install or update (the night leg's `disk` refusal). Not measured: how fast a real box fills. **Needs:** a decision on what the box keeps (the running image + the undo's previous one) and a clean-up after the undo window closes and at remove. `audits/more-night-apps-2026-09-30/box/T2-images-removed-by-name.txt` | **READY — rank P2-MEDIUM; owner: CC (controller); the retention rule needs a word** | -| **R-737** | **[P3-LOW] wger's app login API answers 500 on a CORRECT password: the template sets no `JWT_PRIVATE_KEY`.** MEASURED 2026-09-30 on the bench (wger 2.6, catalog template): `POST /allauth/app/v1/auth/login` with the right admin password → 500, `ValueError: Unable to load PEM file … MalformedFraming` (`allauth/core/internal/jwkkit.py`); a wrong password → 400 JSON (control). wger reads `JWT_PRIVATE_KEY` from the environment (`settings/main.py:109`) and the template sets none. The web login works. Not measured: whether wger's mobile app uses this route (likely — it is the headless API „consumed by the Flutter app", `wger/urls.py`). **Needs:** a generated key (an RSA PEM — the controller's generators do not make one today) or the page saying the mobile app cannot log in. `audits/more-night-apps-2026-09-30/B/wger-probe.txt` | **READY — rank P3-LOW; owner: CC (catalog)** | +| **R-736** | **[P2-MEDIUM] Removing an app or updating it never deletes the image it pulled, so a box's Docker disk fills with old images until an install or an update is refused.** MEASURED 2026-09-30 on scratch guest 9202: a wger install was refused „Docker volume below reserved buffer (4.3G free, reserve 6.8G of 68G)" — correctly; the disk held **84 images, 57 GB, 53 GB of them used by no container** (`docker system df`), left by earlier drills' installs, updates and removals through the product. Read from source: the remove runs `docker compose down --rmi local --volumes` (`felhom-controller` `stacks/delete.go:291`); `--rmi local` removes only images with no custom tag, i.e. never an image pulled by a registry tag. No image clean-up exists anywhere else in the controller (searched: `rmi`, `image prune`, `ImageRemove`). A guarded Update leaves the previous version's image too (it is also the undo's image — keeping the LAST one is deliberate; keeping every one is not decided anywhere). **A household box follows the same path more slowly:** every automatic step adds an image, and the refusal lands on the next install or update (the night leg's `disk` refusal). Not measured: how fast a real box fills. **Needs:** a decision on what the box keeps (the running image + the undo's previous one) and a clean-up after the undo window closes and at remove. `audits/more-night-apps-2026-09-30/box/T2-images-removed-by-name.txt` **-- BUILT 2026-09-30 late (decision 53, option A): controller v0.284.2** (0.284.0/0.284.1 never floored — found live on 9202: the Remove button runs `RemoveStack`, not `DeleteStack`; `docker image ls` without `-a` hides the untagged digest-pulled images). Keep set box-wide at delete time; exact id; no pass while an update runs; one summary line per pass; a one-time sweep. **Measured:** 9202 images 26.6 → 5.7 GB (26 deleted), demo-hp 24.3 → 13.5 GB (24), the N100 6.15 → 6.07 GB (1); every app stayed healthy; a Remove deleted the app's images and kept `postgres:18-alpine`/`redis:7-alpine` that paperless uses. Old controller images stay (R-745). The undo after a clean-up is proven at unit level only (the previous image is in the keep set; red-proofed), not by a live failing update. `audits/night-rulings-2026-09-30/` | **CLOSED 2026-09-30 — controller v0.284.2, floor 0.284.2** | +| **R-737** | **[P3-LOW] wger's app login API answers 500 on a CORRECT password: the template sets no `JWT_PRIVATE_KEY`.** MEASURED 2026-09-30 on the bench (wger 2.6, catalog template): `POST /allauth/app/v1/auth/login` with the right admin password → 500, `ValueError: Unable to load PEM file … MalformedFraming` (`allauth/core/internal/jwkkit.py`); a wrong password → 400 JSON (control). wger reads `JWT_PRIVATE_KEY` from the environment (`settings/main.py:109`) and the template sets none. The web login works. Not measured: whether wger's mobile app uses this route (likely — it is the headless API „consumed by the Flutter app", `wger/urls.py`). **Needs:** a generated key (an RSA PEM — the controller's generators do not make one today) or the page saying the mobile app cannot log in. `audits/more-night-apps-2026-09-30/B/wger-probe.txt` **-- FIXED 2026-09-30 late: catalog `45d8482`** — the start command makes the RSA pair once with wger's own `manage.py generate-jwt-keys`, keeps it 0600 on wger's data volume, loads it. Bench (2.7): right password 200 + an access token that reads the API; wrong password 400 (allauth's answer, not 401); the key survives a restart. Not proven on a box. `audits/night-rulings-2026-09-30/` | **CLOSED 2026-09-30 — catalog `45d8482`** | | **R-738** | **[P2-MEDIUM] Every wger update that brings database migrations leaves wger broken, and the guarded Update reports `done`.** MEASURED 2026-09-30 on scratch guest 9202 (drill catalog): wger 2.6 → 2.7 through the product's guarded Update ended `done` in 72.8 s (health: the front page answers); the web login then answered **500** (`no such column: core_userprofile.time_zone`), still 500 a minute later, **12 migrations unapplied** (`manage.py showmigrations`). Cause, read from the image: `entrypoint.sh` runs `manage.py migrate` only when `DJANGO_PERFORM_MIGRATIONS=True`; the template does not set it (a fresh install works because `wger bootstrap` builds a new database). The harness caught it: the box verdict is `failed`, and no ladder entry was written. No box reporting to the hub runs wger (hub `/apps`, same day). **Also a product gap:** the guarded Update's health check cannot see an app whose front page serves while its data is unusable — the fixture's read-back is what saw it. **Fix:** `DJANGO_PERFORM_MIGRATIONS=True` in the template (the image's own switch for it), then the step again on both venues. `audits/more-night-apps-2026-09-30/box/wger/` **-- FIXED 2026-09-30:** catalog `7a4ff48` sets `DJANGO_PERFORM_MIGRATIONS=True`; the same 2.6 → 2.7 step then ran the migrations (`Applying …` in the log) and read back on 9202 (46.2 s) and on the bench (proven, 0 kills); published `4ad32aa`. The product-side gap (a health check that sees only the front page) remains, as a note: the fixture's read-back caught it, the box could not. | **CLOSED 2026-09-30 — catalog `7a4ff48` + `4ad32aa`** | -| **R-739** | **[P3-LOW] The test bench cannot run wanderer at all: its web server calls the database at the PUBLIC name `https://.`, which the bench has no name or TLS for.** MEASURED 2026-09-30 (more-night-apps brief, Part B): on bench 9401 the catalog template (v0.20.0) came up `wanderer-db` healthy, `wanderer-search` healthy, `wanderer` **unhealthy** for 12 min, every page 500 („Error 0: Something went wrong"); `PUBLIC_POCKETBASE_URL=https://hike-db.gate.invalid`. So the harness can only ever answer `inconclusive` at FROM for wanderer, never about an update. Its step `v0.20.0 → v0.21.0` (web + db) and meilisearch `v1.36 → v1.54` stay untested; **the brief's question — does the search index survive meilisearch's move or is it rebuilt — is NOT measured.** **Needs:** a bench venue that gives the stack the DB name (an `extra_hosts` + plain-http override in the bench's render only, stated in the verdict), or wanderer proven on the box venue alone with the operator's word. `audits/more-night-apps-2026-09-30/B/wanderer-probe.txt` | **READY — rank P3-LOW; owner: CC (catalog harness)** | -| **R-740** | **[P2-MEDIUM] A security fix that upstream ships under the SAME tag (`postgres:18-alpine`, `redis:7-alpine`, `mariadb:12.3` …) reaches NO box — not at night, and not by the household's Update button either, because the catalog never records a re-test of a tag at a new digest.** MEASURED 2026-09-30 (more-night-apps brief, Part C). **(1) The night leg, from source** (`felhom-controller` `d48da6c`): an app at the head tag whose running digest is older than the ladder's tested one reads Behind (`stacks/updateorder.go:86–111` `digestBehind`), but `legCandidate` finds no entry whose `from` is its pin and skips it — `unattended.go:433–435`, `return LegSkipNoTestRecord // at the head, behind only by something no step records`. A unit walk on a scratch copy of the controller confirms it: order Behind → the leg presses nothing, `skipped — no_test_record`; the household's Update button in the same state ends `done` and runs the tested digest (`audits/more-night-apps-2026-09-30/C/C2-*`). **(2) But the catalog never produces that state for a floating tag:** the only writer refuses a step that moves no image (`upgrade-test.py:919`, „--move: nothing moves"), and no ladder in the catalog has an entry with `from == to` or two entries with the same `to` (`C/C4-same-tag-retest.txt`). So the tested digest of `postgres:18-alpine` stays the one of the day the step was tested, a box installed since runs that digest, and the badge reads „Naprakész" while upstream has moved. **(3) How often it matters:** of the 11 distinct floating lines in the catalog today (26 services), the 8 on Docker Hub were pushed 9, 9, 9, 9, 12, 12, 30 and 105 days ago — **7 of 8 within 30 days** (`C/C3-floating-repush.txt`; a push is not proof that the image content changed; ghcr gives no date). Today every tested digest still equals what the registry serves (the tests came after the pushes). Only a box installed BEFORE a step's test can read Behind by digest; the demo boxes have none (`C/C1-digests-demo-hp-9201.txt`, 18 of 18 equal). **(4) Decision 30's cost line says the older image runs „until someone (or the automatic leg) presses Update"** — the automatic leg never does, and the manual press moves the digest only in the legacy case above. **(5) What the box would do if the catalog wrote a same-tag re-test** (unit walk): the leg PRESSES it with no controller change — the newest entry whose `from` is the pin is taken, the update renders the new tested digest, the next night reads Current. So option (a)'s cost is in the catalog. **Needs: a decision (STATUS, 2026-09-30).** | **WAITING-ON-OPERATOR — rank P2-MEDIUM; owner: VIKTOR decides, CC builds** | -| **R-741** | **[P3-LOW] For a few seconds after a fresh install, an `after_install` app answers its PUBLIC default login through the household's front door — the box changes the password only after the app already serves.** MEASURED 2026-09-30 on scratch guest 9202 (calibre-web, controller 0.283.1): polling `GET /opds` with `admin:admin123` (the image's README default) through traefik once a second from the deploy press: 404 until the route existed, **200 at 16:42:06**, 401 from 16:42:07 on (`after_install` done). The first install the same day logged the app started at 15:02:36 and `after_install … done` at 15:02:54 — a fixture probe at 15:02:52 still saw the new password refused, so that window was up to ~18 s. Decision 45 („no app is published with a login a stranger knows") holds after the window, not during it. Who can reach it: anyone who can resolve the app's name in those seconds; calibre-web is not behind the setup gate. Applies to every `after_install` app (calibre-web, mealie, wger, bookstack, claper, …) — not measured for the others. **Needs:** the route published only after `after_install` succeeds (the gate's own route hold, or traefik labels applied after), or the app behind the setup gate until then. `audits/more-night-apps-2026-09-30/A/A3-calibre-default-login-window.txt` | **READY — rank P3-LOW; owner: CC (controller)** | +| **R-739** | **[P3-LOW] The test bench cannot run wanderer at all: its web server calls the database at the PUBLIC name `https://.`, which the bench has no name or TLS for.** MEASURED 2026-09-30 (more-night-apps brief, Part B): on bench 9401 the catalog template (v0.20.0) came up `wanderer-db` healthy, `wanderer-search` healthy, `wanderer` **unhealthy** for 12 min, every page 500 („Error 0: Something went wrong"); `PUBLIC_POCKETBASE_URL=https://hike-db.gate.invalid`. So the harness can only ever answer `inconclusive` at FROM for wanderer, never about an update. Its step `v0.20.0 → v0.21.0` (web + db) and meilisearch `v1.36 → v1.54` stay untested; **the brief's question — does the search index survive meilisearch's move or is it rebuilt — is NOT measured.** **Needs:** a bench venue that gives the stack the DB name (an `extra_hosts` + plain-http override in the bench's render only, stated in the verdict), or wanderer proven on the box venue alone with the operator's word. `audits/more-night-apps-2026-09-30/B/wanderer-probe.txt` **-- 2026-09-30 late: the bench CAN run wanderer now** — `upgrade-test.py` `BENCH_ENV_OVERRIDES` points `PUBLIC_POCKETBASE_URL` at the database container on the bench only (the only address the web image reads, measured in v0.20.0 and v0.21.0), recorded in every verdict: web 200, sign-up (`PUT /api/v1/user`) 200, login 200. **The meilisearch question, answered:** v1.54.2 REFUSES v1.36's database („Your database version (1.36.0) is incompatible”); with `MEILI_UPGRADE_DB=true` it upgrades in place and all three indexes (actors, lists, trails) are there — so wanderer's step needs that switch in the template first. Not done: a fixture (creating a list answered PocketBase's „Failed to create record” — likely a rule for unverified users), whether a household's record survives in the index, the step itself. `audits/night-rulings-2026-09-30/` | **NARROWED — the bench runs it; the step needs a fixture and `MEILI_UPGRADE_DB`; owner: CC** | +| **R-740** | **[P2-MEDIUM] A security fix that upstream ships under the SAME tag (`postgres:18-alpine`, `redis:7-alpine`, `mariadb:12.3` …) reaches NO box — not at night, and not by the household's Update button either, because the catalog never records a re-test of a tag at a new digest.** MEASURED 2026-09-30 (more-night-apps brief, Part C). **(1) The night leg, from source** (`felhom-controller` `d48da6c`): an app at the head tag whose running digest is older than the ladder's tested one reads Behind (`stacks/updateorder.go:86–111` `digestBehind`), but `legCandidate` finds no entry whose `from` is its pin and skips it — `unattended.go:433–435`, `return LegSkipNoTestRecord // at the head, behind only by something no step records`. A unit walk on a scratch copy of the controller confirms it: order Behind → the leg presses nothing, `skipped — no_test_record`; the household's Update button in the same state ends `done` and runs the tested digest (`audits/more-night-apps-2026-09-30/C/C2-*`). **(2) But the catalog never produces that state for a floating tag:** the only writer refuses a step that moves no image (`upgrade-test.py:919`, „--move: nothing moves"), and no ladder in the catalog has an entry with `from == to` or two entries with the same `to` (`C/C4-same-tag-retest.txt`). So the tested digest of `postgres:18-alpine` stays the one of the day the step was tested, a box installed since runs that digest, and the badge reads „Naprakész" while upstream has moved. **(3) How often it matters:** of the 11 distinct floating lines in the catalog today (26 services), the 8 on Docker Hub were pushed 9, 9, 9, 9, 12, 12, 30 and 105 days ago — **7 of 8 within 30 days** (`C/C3-floating-repush.txt`; a push is not proof that the image content changed; ghcr gives no date). Today every tested digest still equals what the registry serves (the tests came after the pushes). Only a box installed BEFORE a step's test can read Behind by digest; the demo boxes have none (`C/C1-digests-demo-hp-9201.txt`, 18 of 18 equal). **(4) Decision 30's cost line says the older image runs „until someone (or the automatic leg) presses Update"** — the automatic leg never does, and the manual press moves the digest only in the legacy case above. **(5) What the box would do if the catalog wrote a same-tag re-test** (unit walk): the leg PRESSES it with no controller change — the newest entry whose `from` is the pin is taken, the update renders the new tested digest, the next night reads Current. So option (a)'s cost is in the catalog. **Needs: a decision (STATUS, 2026-09-30).** **-- BUILT 2026-09-30 late (decision 52, option A):** catalog `6a3ead9` — a re-test entry (`from` == `to`, `digest_from`, `box_evidence`), refused by the gates with no new digest, a digest the registry stopped serving, no box proof, or a `digest_from` that is not the previous digest (8 decoys, seen red); `upgrade-test.py --retest`; the ONE command `scripts/retest-floating.py` (`--dry-run`, `--engines-only`). **Proven end to end on 9202:** docmost at an older `redis:7-alpine` digest, the re-test, „run tonight's chain now”, the leg pressed it (`step ended done after 95.0 s`), the new digest runs, data read back, badge current. **Run for real:** no database/redis line differs today (two exact tags do — R-743). Not a cron job (runbook `runbooks/monthly-floating-retest.md` says why); who presses it monthly is the operator's word (STATUS). `audits/night-rulings-2026-09-30/` | **CLOSED 2026-09-30 — built (catalog `6a3ead9`); the monthly run is a standing step** | +| **R-741** | **[P3-LOW] For a few seconds after a fresh install, an `after_install` app answers its PUBLIC default login through the household's front door — the box changes the password only after the app already serves.** MEASURED 2026-09-30 on scratch guest 9202 (calibre-web, controller 0.283.1): polling `GET /opds` with `admin:admin123` (the image's README default) through traefik once a second from the deploy press: 404 until the route existed, **200 at 16:42:06**, 401 from 16:42:07 on (`after_install` done). The first install the same day logged the app started at 15:02:36 and `after_install … done` at 15:02:54 — a fixture probe at 15:02:52 still saw the new password refused, so that window was up to ~18 s. Decision 45 („no app is published with a login a stranger knows") holds after the window, not during it. Who can reach it: anyone who can resolve the app's name in those seconds; calibre-web is not behind the setup gate. Applies to every `after_install` app (calibre-web, mealie, wger, bookstack, claper, …) — not measured for the others. **Needs:** the route published only after `after_install` succeeds (the gate's own route hold, or traefik labels applied after), or the app behind the setup gate until then. `audits/more-night-apps-2026-09-30/A/A3-calibre-default-login-window.txt` **-- FIXED 2026-09-30 late: controller v0.284.x** — an `after_install` app is installed HELD behind the setup gate's door until the login is replaced (or the household says it changed it). **Live on 9202, as a stranger:** calibre-web 0 of 192 default-login tries got in (hold before the first start 20:36:32; opened by after_install 20:37:18, 17 s after the app was up; then 401); mealie 0 of 97 (then mealie's own lock — R-747). The positive control with the generated password was not obtained (calibre-web: a backup stopped the app at that moment; mealie: the lock). Red-proofs RP-IH1..4. `audits/night-rulings-2026-09-30/` | **CLOSED 2026-09-30 — controller v0.284.2** | | **R-742** | **[P3-LOW] zipline 4.8.0 cannot be reached from 4.6.1 in one step: it refuses to start until the database has run the release before it.** MEASURED 2026-09-30 on both venues, the same way: 4.6.1 → 4.8.0 — the new container restarts (exit 1) with `Error: cannot safely migrate from prisma to drizzle: expected migration 20260508022000_add_file_folder_created_at_index was not applied. To resolve this, repair the database with the previous (latest before this) Zipline release before upgrading` (bench `to-full.log`, 15×). **On box 9202 the product's guarded Update saw it unhealthy after 1 m 30 s and UNDID it by itself in 20 s — the previous version back on the data from before the update** (`box/zipline/step.txt`): the undo worked on a real upstream failure, not a drill. Bench verdict `failed`, box `undone`; nothing written. **Also found:** the zipline fixture had two faults, both fixed (`/` answers 301 → wait on `/api/healthcheck`; a wrong-password control first trips its login limit → 429 on the right one; the right password now goes first). **Needs:** the ladder climbs 4.6.1 → 4.7.x → 4.8.0 (two tested steps — the ladder exists for exactly this). `audits/more-night-apps-2026-09-30/bench/apps/zipline/`, `box/zipline/` **-- DONE 2026-09-30 (evening):** the two steps, each proven on both venues — 4.6.1 → 4.7.0 (catalog `a9700e2`; box 103.6 s, bench 0 kills) and 4.7.0 → 4.8.0 (`fb87030`; box 32.8 s, bench 0 kills). | **CLOSED 2026-09-30 — catalog `a9700e2` + `fb87030`** | +| **R-743** | **[P3-LOW] Exact version tags are re-pushed under the same name too — not only floating lines.** MEASURED 2026-09-30 by `retest-floating.py --dry-run` on the live catalog (`6a3ead9`): `nextcloud:34.0.4-apache` (tested `a5ace30c…`, registry `37b10988…`) and `lscr.io/linuxserver/sonarr:4.0.20` (tested `a5c1a5fe…`, registry `f247545d…`) — the registry now serves another build under a tag the ladder tested. No database or redis line differed. Decision 52 starts with the engine lines, so these were NOT re-tested (`--engines-only`). linuxserver rebuilds its images weekly under the same tags, so every linuxserver app will show up here. **Needs:** a word on whether the monthly run covers every app (the command does; `--engines-only` is the switch) or only engines. `audits/night-rulings-2026-09-30/A/` | **WAITING-ON-OPERATOR — rank P3-LOW; owner: VIKTOR decides, CC runs** | +| **R-744** | **[P3-LOW] outline's fixture cannot seed outline 1.10.1: no `csrfToken` cookie after `installation.create`.** MEASURED 2026-09-30 on both venues (bench and 9202, the end-to-end re-test's first attempt): `POST /api/installation.create` answered 302 with the session, and `GET /home` set no `csrfToken` cookie (1.9.1 did; the 1.9.1 → 1.10.1 step read back that afternoon because its read-back uses the API key made at 1.9.1). So the NEXT outline step, and any re-test of outline, stops at C1 with „no csrfToken cookie from GET /home". **Needs:** the fixture reads outline 1.10.1's CSRF the way its page does (measure first). `audits/night-rulings-2026-09-30/A/e2e/*outline-attempt*` | **READY — rank P3-LOW; owner: CC (catalog harness)** | +| **R-745** | **[P3-LOW] Old CONTROLLER images are never deleted: ~50 versions on each demo box.** MEASURED 2026-09-30 after v0.284.2's one-time sweep: every image no container uses on demo-hp and on the N100 is a `felhom-controller` tag (0.201.0 … 0.283.1); the N100 still reads 3.6 GB reclaimable. Decision 53 covers APP images, and the sweep leaves the controller's own images alone on purpose (the self-update may need the previous one). A release is ~150 MB and there are several a day. **Needs:** the same rule for the controller — keep the running and the previous tag — as a decision (the self-update's rollback reads which image?). `audits/night-rulings-2026-09-30/E/` | **READY — rank P3-LOW; owner: CC (controller); the rule needs a word** | +| **R-746** | **[P3-LOW] `image_digest.resolve` ignores a `@digest` in its argument — it answers the TAG's current digest.** MEASURED 2026-09-30: `resolve('redis:7-alpine@sha256:000…0')` returned `sha256:858f…` (the tag's), while a manifest request for that digest answers 404. Every gate today passes it a plain tag, so no gate is wrong; a caller that passes `ref@digest` to ask "is THIS digest still served" gets a false yes. **Needs:** refuse a digest-carrying ref, or ask for the manifest by digest (with a test). `scripts/image_digest.py` | **READY — rank P3-LOW; owner: CC (catalog)** | +| **R-747** | **[P3-LOW] A stranger can lock the household out of mealie with five wrong logins.** MEASURED 2026-09-30 on 9202 (the R-741 proof): after the install hold opened, a stranger's default-login tries were refused (401) and after five of them mealie answered 423 (locked) to every login — the generated, correct password included. mealie's own brute-force guard, on an app published on the internet; the setup gate and the install hold do not cover an app after its first setup. Not measured: how long the lock lasts. **Needs:** measure the lock's length; decide whether the page tells the household what to do. `audits/night-rulings-2026-09-30/C/C3-mealie-poll.txt` | **READY — rank P3-LOW; owner: CC** | +| **R-748** | **[P3-LOW] The register-shape gate skipped every row whose id has a letter suffix — so R-88a, R-88b and R-209a were never shape-checked, and its count read 3 short.** FOUND 2026-09-30 (late) while counting the register: `register_shape_gate.py` matched `R-\d+` only; the brief's „the reviewer's regex undercounted by 3” is the same three rows. Fixed the same session: `R-\d+[a-z]?`; decoy `suffix-row-eaten-state` (a suffixed row with its state cell eaten) seen passing with the old pattern and convicted with the new. The register is **382** rows by either count now. | **CLOSED 2026-09-30 — `scripts/register_shape_gate.py`** |