bc47dd4ef9
gates / gates (push) Successful in 18s
The operator looked at demo-felhom and found OpenGist - up 15 hours, running exactly the catalog pin, showing no badge at all. 09-update-architecture.md had recorded that as an accepted limitation the day before: 'the fleet view fills in gradually'. On a quiet box gradually means never, and a feature that fills itself in on an event nobody triggers is, on the quiet installations, not shipped. That limitation row is now struck with the reason kept. The living document gains slice 1b, the two admission rules of the backfill (it never overwrites, and it refuses to seed a partial observation because the badge reads a service-count mismatch as BEHIND), and the note that the same field having two writers with two different admission rules is deliberate. Live evidence added: all nine apps already had records by the time 0.234.0 was ready, so the natural fleet state could no longer exercise the new code - said plainly rather than papered over. The pre-0.233.0 shape was recreated on demo-hp by stripping two records; the backfill re-seeded exactly those two with digests matching independently-read ground truth and left the other seven alone. The refusal half was deliberately NOT staged live: it needs a degraded app, and manufacturing one risks the false-customer-email class that already cost 61 mails (R-330). Unit-tested with a red-proof, and recorded as unproven-live. R-457: a test that hardcodes a date and asserts an age derived from it is green only on the day it is written. Mine was, and it went red overnight. Six other files carry both a date literal and time.Now() - named as candidates, not accused.
283 lines
15 KiB
Markdown
283 lines
15 KiB
Markdown
# VALIDATION — update arc slices 1 & 2, live on demo-hp (2026-09-02)
|
||
|
||
**Controller `0.233.0`, guest 9201 on `demo-hp` (Tier 0 — disposable).** Evidence copied off the box
|
||
at the end of the phase that produced it, not at the end of the session.
|
||
|
||
**Method: endpoint/production-path level.** `claude-in-chrome` is not available on DooPlex.
|
||
**Which path was used, stated exactly: the BOOT RECONCILER**, `bootrecon.Run → StackProvider.StartStack
|
||
→ compose up -d → recordInstalledImages` — a REAL production caller, the same one
|
||
`SPIKE-app-update-2026-09-01` §2 variant 1c-ii used, and no hand-set state anywhere.
|
||
**Why not the customer's Restart button: §4.0 — the dashboard login was lost to a mistake of mine
|
||
for part of the run, and the record half was measured before it was recovered.**
|
||
|
||
---
|
||
|
||
## 1. Deployed version
|
||
|
||
```
|
||
$ pct exec 9201 -- docker ps --filter name=felhom-controller --format '{{.Image}} {{.Status}}'
|
||
gitea.dooplex.hu/admin/felhom-controller:0.233.0 Up 19 seconds (healthy)
|
||
```
|
||
|
||
Previous: `0.232.0`. Image digest `sha256:df5940ccf5a548ceee9065a2cc55a467f8941c636138441e0d77e320dc2023ab`.
|
||
|
||
## 2. The record — PROVEN LIVE, on a single-service AND a multi-service app
|
||
|
||
### 2.0 Before — nine deployed apps, ZERO with a record
|
||
|
||
```
|
||
bentopdf deployed=1 installed_images=0 paperless-ngx deployed=1 installed_images=0
|
||
bookstack deployed=1 installed_images=0 privatebin deployed=1 installed_images=0
|
||
calibre-web deployed=1 installed_images=0 romm deployed=1 installed_images=0
|
||
docmost deployed=1 installed_images=0
|
||
kimai deployed=1 installed_images=0
|
||
opengist deployed=1 installed_images=0
|
||
```
|
||
|
||
**That is the legacy case, and it is the state the badge must render NOTHING for.**
|
||
|
||
### 2.0b Ground truth, read from the containers BEFORE anything was touched
|
||
|
||
```
|
||
bentopdf ref=ghcr.io/alam00000/bentopdf:v2.8.6 rd=…@sha256:eaeea1e447205a79cb61d7efdc6966f37311dc1bc9c36a3a5c897bf79107c2c3
|
||
bookstack ref=lscr.io/linuxserver/bookstack:26.05.2 rd=…@sha256:3db259db582808ab498d49ae96b0a63f935d9cf3635c9d5bd8b8815c6ff1f8a1
|
||
bookstack-db ref=mariadb:12.3 rd=…@sha256:a02fe89cb597d4375812b2eac90cf9d0775d4686daa7f7cc750ebbcad7525bbc
|
||
```
|
||
|
||
**This is the discriminator.** Every digest below is compared against these, taken independently.
|
||
|
||
### 2.1 Single service — `bentopdf`
|
||
|
||
Staged exactly as the spike stages a boot orphan: `docker rm -f bentopdf` (chosen because it has **no
|
||
database, no volume and no data of any kind**), then the controller restarted so the reconciler runs.
|
||
`desired_state: running` was left untouched. **Nothing else was staged; the reconciler selected the app
|
||
on its own.**
|
||
|
||
```
|
||
18:27:32 main.go:2162: [INFO] [bootrecon] boot window: fleet settled after 30s — sweeping
|
||
18:27:32 bootrecon.go:259: [INFO] [bootrecon] Boot reconciliation: 1 boot-orphaned app(s) found: [bentopdf]
|
||
18:27:32 manager.go:1066: [INFO] [stacks] Starting stack: bentopdf
|
||
18:27:32 manager.go:1081: [INFO] [stacks] Stack bentopdf started successfully (took 0.3s)
|
||
18:27:32 installed.go:408: [INFO] [stacks] installed-images bentopdf: recorded 1 service(s) (bentopdf=ghcr.io/alam00000/bentopdf:v2.8.6 (sha256:eaeea1e44720…))
|
||
```
|
||
|
||
`/opt/docker/stacks/bentopdf/app.yaml` after:
|
||
|
||
```yaml
|
||
installed_images:
|
||
bentopdf:
|
||
ref: ghcr.io/alam00000/bentopdf:v2.8.6
|
||
digest: sha256:eaeea1e447205a79cb61d7efdc6966f37311dc1bc9c36a3a5c897bf79107c2c3
|
||
at: "2026-09-02T18:27:32Z"
|
||
```
|
||
|
||
**Digest MATCHES §2.0b exactly.**
|
||
|
||
### 2.2 Multi-service — `bookstack`. **The case that matters.**
|
||
|
||
**One entry for the whole stack is the wrong outcome, and only a multi-container app can tell.**
|
||
|
||
Same staging (`docker rm -f bookstack bookstack-db`; both volumes are NAMED —
|
||
`bookstack_bookstack_config`, `bookstack_bookstack_db_data` — and were verified present, so no data
|
||
was at risk), then the controller restarted.
|
||
|
||
```
|
||
18:28:48 bootrecon.go:259: [INFO] [bootrecon] Boot reconciliation: 1 boot-orphaned app(s) found: [bookstack]
|
||
18:28:48 manager.go:1066: [INFO] [stacks] Starting stack: bookstack
|
||
18:28:54 installed.go:408: [INFO] [stacks] installed-images bookstack: recorded 2 service(s) (bookstack=lscr.io/linuxserver/bookstack:26.05.2 (sha256:3db259db5828…), bookstack-db=mariadb:12.3 (sha256:a02fe89cb597…))
|
||
18:28:54 bootrecon.go:303: [INFO] [bootrecon] Boot reconciliation complete: 1 app(s) recovered in 1 attempt(s)
|
||
```
|
||
|
||
```yaml
|
||
installed_images:
|
||
bookstack:
|
||
ref: lscr.io/linuxserver/bookstack:26.05.2
|
||
digest: sha256:3db259db582808ab498d49ae96b0a63f935d9cf3635c9d5bd8b8815c6ff1f8a1
|
||
at: "2026-09-02T18:28:54Z"
|
||
bookstack-db:
|
||
ref: mariadb:12.3
|
||
digest: sha256:a02fe89cb597d4375812b2eac90cf9d0775d4686daa7f7cc750ebbcad7525bbc
|
||
at: "2026-09-02T18:28:54Z"
|
||
```
|
||
|
||
**TWO entries, keyed by COMPOSE SERVICE NAME, both digests matching §2.0b exactly.**
|
||
|
||
**A NOTE ON WHY IT TOOK TWO PASSES, because it is a real fact about the reconciler and not a mishap.**
|
||
The first pass removed only the `bookstack` app container and left `bookstack-db` running; the
|
||
reconciler did **not** select bookstack — it found `[bentopdf]` alone. A stack with one live member is
|
||
not a boot orphan to it. Removing the DB container as well made the whole stack orphaned and it was
|
||
selected on the next pass. Both apps are running and healthy at the end (§5).
|
||
|
||
### 2.3 The encrypted secrets survived the write — checked, not assumed
|
||
|
||
`bookstack`'s `app.yaml` holds two encrypted values. Before and after the record was written, both
|
||
`ENC:` strings are **byte-identical**, and `deployed`, `deployed_at`, `locked_fields` and
|
||
`desired_state` are unchanged. This is the copy-and-overlay property (R-100) holding under a new field.
|
||
|
||
## 3. `catalog_since` reached the box by the NORMAL 15-minute sync
|
||
|
||
No force, no hand-edit:
|
||
|
||
```
|
||
$ grep -n catalog_since /opt/docker/stacks/bookstack/.felhom.yml
|
||
13:catalog_since: "2026-07-18"
|
||
```
|
||
|
||
## 4. The badge render — **PROVEN LIVE on both surfaces, three of the four states**
|
||
|
||
### 4.0 A false diagnosis of my own, corrected here rather than buried
|
||
|
||
The first pass reported "the vaulted dashboard password is stale on both demo boxes", with the
|
||
controller's own `[WARN] Failed login` quoted as the discriminator. **The password was fine. The
|
||
extraction was wrong.** `~/.config/credentials` quotes its values with **single** quotes and the `sed`
|
||
used stripped only double quotes, so the literal `'` characters were sent as part of the password.
|
||
The operator said so, one retry with `sed "s/^['\"]//;s/['\"]$//"` returned **HTTP 302 +
|
||
`felhom_session`**, and everything below followed.
|
||
|
||
**This is exactly the trap the `credentials-file-values-are-quoted` memory records** — it was applied
|
||
half-way. The controller's log was a true observation and a misleading one: `Failed login` proves the
|
||
BYTES did not match the hash; it says nothing about whose fault that is. **A discriminator that
|
||
separates "wrong password" from "wrong host header" does not separate "wrong password" from "wrong
|
||
password handling", and I read it as if it did.** Register row R-453 was opened on the wrong premise
|
||
and is corrected there.
|
||
|
||
### 4.1 „Naprakész" — quoted from the live pages
|
||
|
||
`GET /apps/bookstack` and `GET /stacks`, session-authenticated, `Host: felhom.enkisfelhom.hu`:
|
||
|
||
```html
|
||
<span class="tag tag-ok" title="Ez az alkalmazás a legfrissebb elérhető változatot futtatja.">Naprakész</span>
|
||
```
|
||
|
||
Byte-identical to the string table in the task. **On `/stacks` it appears exactly TWICE** — the two
|
||
apps that have a record — while the **seven other deployed apps render NOTHING**. That is the
|
||
no-record case, observed live and not inferred.
|
||
|
||
### 4.2 „Frissítés elérhető — N napja" — the age comes from `catalog_since`, live
|
||
|
||
To reach the behind state the badge needs the template to pin something the container is not running.
|
||
Staged on **`bentopdf`** — the app with no database, no volume and no data of any kind — by editing
|
||
**only** its `docker-compose.yml` tag `v2.8.6 → v2.8.5` and touching nothing else. **The container was
|
||
never restarted and no `up -d` ran**; the file is the badge's input, and this is the same shape the
|
||
syncer produces on its own. `ScanStacks` (2-minute cadence) picked it up:
|
||
|
||
```html
|
||
<span class="tag tag-warn" title="Újabb változat érhető el ehhez az alkalmazáshoz. A frissítés indításához nyomd meg a Frissítés gombot.">Frissítés elérhető — 52 napja</span>
|
||
```
|
||
|
||
**52 napja is arithmetic on a real catalog value, not a placeholder:** bentopdf's `catalog_since` is
|
||
`2026-07-12` and the run is 2026-09-02 — 52 days. It rendered on the app page **and** the app list.
|
||
|
||
**REVERTED IMMEDIATELY, and verified byte-identical:** `sha256` before and after both
|
||
`39679e28cdd6ee8f46e359290b4d631cf234a1cfdae374cd84c8fe7588870ed1`, `diff` empty, container still
|
||
`ghcr.io/alam00000/bentopdf:v2.8.6 Up 17 minutes (healthy)` throughout. The badge then returned to
|
||
„Naprakész" with `napja` count **0**.
|
||
|
||
**GRADED HONESTLY: the compose file was placed by hand, not by the syncer.** What is proven is the
|
||
render — compose file → `ScanStacks` → `TemplateImages` → `updateBadge` → page — with the file
|
||
carrying exactly what a sync would have written. The syncer's own half is separately measured in
|
||
`SPIKE-app-update-2026-09-01` §3.
|
||
|
||
### 4.3 ASCII-fragment counts, with a positive and a negative control
|
||
|
||
Accented text is never grepped directly here. `grep -oF`, so no `.` is a wildcard — the correction the
|
||
spike had to make on itself.
|
||
|
||
| fragment | `/stacks` (current) | `/apps/bookstack` | `/apps/docmost` (legacy) | `/stacks` (behind) | `/apps/bentopdf` (behind) |
|
||
|---|---|---|---|---|---|
|
||
| `Naprak` | **2** | **1** | **0** | 1 | 0 |
|
||
| `napja` | 0 | 0 | 0 | **1** | **1** |
|
||
| `52 napja` | 0 | 0 | 0 | **1** | **1** |
|
||
| `legfrissebb` (hover) | 2 | 1 | 0 | — | — |
|
||
| `BookStack` (positive control) | 1 | 4 | 0 | — | — |
|
||
| `Docmost` (positive control) | 1 | 0 | 5 | — | — |
|
||
| `zzz-never-present` (**negative control**) | **0** | **0** | **0** | **0** | **0** |
|
||
| `Nem-karbantartott-XYZ` (**negative control**) | **0** | **0** | **0** | — | — |
|
||
|
||
`/apps/docmost` is the load-bearing column: a deployed app with no record renders **no badge at all**,
|
||
live. **Absent is UNKNOWN, and it is not rendered as current.**
|
||
|
||
### 4.4 Nothing about updating changed — checked on the live page
|
||
|
||
In the behind state, `/stacks` still carries exactly one of each for bentopdf:
|
||
|
||
```
|
||
stackAction(event, 'bentopdf', 'update') ×1
|
||
stackAction(event, 'bentopdf', 'restart') ×1
|
||
stackAction(event, 'bentopdf', 'stop') ×1
|
||
```
|
||
|
||
The badge is wired to nothing.
|
||
|
||
### 4.5 The one state NOT reachable live
|
||
|
||
**„Frissítés elérhető" with no age** — it needs an app whose `catalog_since` is absent, malformed or
|
||
future-dated, and all 53 catalog apps now carry a valid one. Covered by `TestGroupF`, which walks
|
||
absent, blank, `tegnap`, `18/07/2026`, `2026-13-45` and a future date.
|
||
|
||
## 4b. v0.234.0 — the startup backfill, PROVEN LIVE 2026-09-03
|
||
|
||
**Why it exists.** §4 of this file, written 2026-09-02, recorded *"the record only appears after the
|
||
next lifecycle action"* as an accepted limitation. The operator looked at **demo-felhom** the next
|
||
morning and found **OpenGist** — up 15 hours, running exactly the catalog pin, showing **no badge at
|
||
all**. On a quiet box "fills in gradually" means "never".
|
||
|
||
### The staging, and why it was needed
|
||
|
||
By the time v0.234.0 was ready, **all nine deployed apps on demo-hp and the one on demo-felhom already
|
||
had records** — the overnight backup cycle had restarted them and v0.233.0's recorder had fired on
|
||
every one (opengist's record is stamped `2026-09-03T00:31:11Z`). **So the natural fleet state could no
|
||
longer exercise the new code**, and saying that plainly matters more than a green log line.
|
||
|
||
The pre-0.233.0 shape was therefore recreated on **demo-hp** (Tier 0): the `installed_images:` block
|
||
was deleted from `privatebin` (1 service) and `romm` (**3** services) — a record, never data — and the
|
||
controller restarted. Ground truth was read from the containers first, independently.
|
||
|
||
### The result
|
||
|
||
```
|
||
09:59:45 installed.go:519: [INFO] installed-images backfill: privatebin recorded 1 service(s) (privatebin=privatebin/pdo:2.0.5 (sha256:8a2cac16eff6…))
|
||
09:59:45 installed.go:519: [INFO] installed-images backfill: romm recorded 3 service(s) (romm=rommapp/romm:5.0.0 (sha256:91f6611eca5a…), romm-db=mariadb:11.4 (sha256:4f1d8d202fcf…), romm-redis=redis:7-alpine (sha256:ff02b58f971e…))
|
||
09:59:45 installed.go:525: [INFO] installed-images backfill: 2 app(s) recorded, 7 already had a record, 0 left unrecorded
|
||
```
|
||
|
||
- **Exactly the two stripped apps were seeded**, and **the other seven were not touched** — the
|
||
never-overwrite rule, observed rather than asserted.
|
||
- **Every digest matches the ground truth** read from the containers beforehand:
|
||
`privatebin/pdo@sha256:8a2cac16eff6caed4dc622e7b3ebd0c0ffcb28dfc0eba0b9c335636552eadac1`,
|
||
`rommapp/romm@sha256:91f6611eca5a4dafc4f4a1d72a1ed7dd66a11375d939f28410dc1d1de0b80b1b`,
|
||
`mariadb@sha256:4f1d8d202fcf7bcb3902f63af09f9c1a050c2922a89652f22abaec0d4f015e83`.
|
||
- The re-seeded records are identical to the ones stripped, apart from `at` — which is correct: it
|
||
records when THIS observation was first made.
|
||
- **`/stacks` on demo-hp then carried „Naprakész" ×9** — one for every deployed app — with the
|
||
negative control `zzz-never-present` at **0** and `napja` at **0** (nothing is behind on that box).
|
||
- On **demo-felhom** the operator's own case renders:
|
||
`<span class="tag tag-ok" …>Naprakész</span>` on `/stacks?filter=running` **and** on `/apps/opengist`.
|
||
|
||
### What was NOT proven live, and why the choice was made
|
||
|
||
**The refusal half — that a partial observation is not seeded — was not staged on a live box.** Doing
|
||
so means stopping one container of a multi-service app to make it degraded, and a degraded app is a
|
||
dead-app alarm candidate; **this project has already paid for 61 false customer e-mails from exactly
|
||
that class (R-330), and manufacturing one to demonstrate a guard is a bad trade.** It is covered by
|
||
`TestGroupG_BackfillRefusesAPartialObservation` **with a companion red-proof** — removing the guard
|
||
makes it fail with *"backfilled 1, want 0"*. Stated as unproven-live rather than left unmentioned.
|
||
|
||
### Teardown
|
||
|
||
The two `app.yaml` backups taken on the box were removed. **Nothing was provisioned** and no app was
|
||
started, stopped or upgraded at any point.
|
||
|
||
## 5. End state — nothing left broken, nothing provisioned
|
||
|
||
```
|
||
$ pct exec 9201 -- docker ps --format '{{.Names}}|{{.Image}}|{{.Status}}' | grep -E 'bookstack|bentopdf'
|
||
bookstack|lscr.io/linuxserver/bookstack:26.05.2|Up 10 seconds (health: starting)
|
||
bookstack-db|mariadb:12.3|Up 16 seconds (healthy)
|
||
bentopdf|ghcr.io/alam00000/bentopdf:v2.8.6|Up About a minute (healthy)
|
||
```
|
||
|
||
All three containers are back on the SAME images they ran before, both named volumes untouched, and
|
||
both apps carry a complete record. **This run provisioned nothing** — no guest, no storage, no hub
|
||
record — so there is no teardown to report on any of the three layers.
|