The household is told: controller v0.264.0 + hub v0.120.0 proven live, floor 0.264.0
gates / gates (push) Successful in 28s
gates / gates (push) Successful in 28s
09 §6.4 parts 2-3 SHIPPED. R-606, R-620, R-646 closed; R-647 (three leftovers) and R-648 (whole-box backup press in the harness) opened. Open rows 335 -> 334. Evidence: audits/undo-fleet-2026-09-23/. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
@@ -1,11 +1,11 @@
|
||||
# REPORT — the undo: bake-off, build (controller v0.263.0 → v0.263.2), live proof
|
||||
# REPORT — the undo reaches the fleet, and the household is TOLD (controller v0.264.0, hub v0.120.0)
|
||||
|
||||
2026-09-23 (afternoon). Repos touched: **felhom-controller** (v0.263.0, v0.263.1, v0.263.2),
|
||||
**felhom.eu** (docs, register, capability map, STATUS, CONTEXT, evidence), **admin/app-catalog-drill**
|
||||
(drill commits, reset to live `main` at the end). **app-catalog-felhom.eu, felhom-agent, hub: untouched.**
|
||||
Architecture read first and named: `documentation/architecture/09-update-architecture.md` (§3 decisions
|
||||
11–20, §4, §6.1, §6.1a, §6.4). Baselines verified live: controller `b9deec19077b`, agent
|
||||
`d9864a94bf62`, felhom.eu `4c92beab8fdb`, catalog `cfcfe5278428` — all as the brief said.
|
||||
2026-09-23 (evening). Repos touched: **felhom-controller** (v0.264.0, `bc27894`), **felhom.eu** (hub
|
||||
v0.120.0 `d3b2848`, manifest `7caa24f`, docs, register, evidence), **admin/app-catalog-drill** (drill
|
||||
commits, reset to live `main` at the end). **felhom-agent, app-catalog-felhom.eu: untouched.**
|
||||
Architecture read and named: `documentation/architecture/09-update-architecture.md` (§3 decision 15,
|
||||
§6.4). Evidence: `documentation/audits/undo-fleet-2026-09-23/README.md`. **Method: endpoint-level**
|
||||
(the endpoints the UI invokes; mails read from the catch-all mailbox; no browser).
|
||||
|
||||
---
|
||||
|
||||
@@ -13,69 +13,104 @@ Architecture read first and named: `documentation/architecture/09-update-archite
|
||||
|
||||
| item | state |
|
||||
|---|---|
|
||||
| rulings 19–20 | **recorded** (`09` §3), commit `5a349d9` |
|
||||
| Part 1 — bake-off | **done in ~40 min** of the 2 h cap; folder copy chosen |
|
||||
| Part 2 — build | **done — but in THREE releases, not one.** v0.263.0 failed its first two live proofs honestly (HELD, data put back). Two defects only the live box could show; each fixed + red-proofed + released: **v0.263.1** (the undo's probe never ran on an app the current probe held `unhealthy`), **v0.263.2** (the "old" `.felhom.yml` was already the new one — it flows in on every catalog sync; now recorded at pin time). 0.263.0/0.263.1 ran only on 9202 and were removed from it. |
|
||||
| the household mail + operator event of decision 15 | **not built** — `09` §6.4 part 2 (R-606). The page line and the hold sentence are built. |
|
||||
| the floor | **not raised** — the operator's question (STATUS) |
|
||||
| immich/nextcloud rate test | **nextcloud deployed and was used** (185 MB MariaDB volume, 300 files through WebDAV); immich not tried |
|
||||
| cut-off copy on vikunja in the bake-off | **could not be cut** (2.9 MB finishes before a kill lands); proven on docmost and romm instead; in the LIVE proof the marker was removed from a vikunja copy mid-update |
|
||||
| Part 0 — floor 0.263.2 | **done**, read back (`00-floor-0.263.2.txt`) |
|
||||
| Part 1 — R-646 startup pass | **done**; live: 3 recorded on 9202, 10 on 9201, 0 skipped |
|
||||
| Part 2 — two events, hub v0.120.0 first | **done**; hub deployed 11:49Z, before any box ran v0.264.0 |
|
||||
| Part 3 — R-606 | **done, with leftovers → R-647** (a reader in the other language than the box sees a HELD sentence in the box's language; the English mail's raw `Note:` line keeps the Hungarian `copy_holds` phrase) |
|
||||
| Part 4 — R-620 | **done** |
|
||||
| Part 5 — live proof | **done**; all proofs passed → **floor raised to 0.264.0** |
|
||||
| "standing apps untouched" | **NOT fully met.** The harness's „Mentés most" is a whole-box backup: on 9201 it stopped and restarted 9 of the 10 standing apps for their volume dumps, twice (a few seconds each). All healthy afterwards, files byte-identical. **R-648** filed. |
|
||||
| the brief's Hungarian undone line | **changed:** it ends „semmi nem veszett el, nincs teendőd." — the brief's „teendője nincs" is the formal „ön" voice; the product speaks „te" everywhere and the voice gate ratchets the formal form (18). |
|
||||
| the customer cooldown | **changed shape:** a NEW customer-side register (`perAppCustomerCooldownEvents`) instead of putting the customer leg of R-389's register to work — that would also have changed `app_start_failed`'s household grain, which nobody asked for (pinned by a test). |
|
||||
| a box-side seed + two page toggles | **added, not in the brief:** the box pushes its own `EnabledEvents` to the hub on every notifications-page save, so the hub migration alone would be undone by the next save. |
|
||||
|
||||
**Claims in the brief that turned out wrong or unmeasured:**
|
||||
1. *"All three apps keep their data only in named volumes"* — true from compose AND on disk for these
|
||||
three; but romm also has two BIND folders (roms, resources), empty throughout — never copied, by rule.
|
||||
2. *"A volume copy with the containers stopped is consistent for both engines"* — **measured true**:
|
||||
docmost (PostgreSQL) and romm (MariaDB) came back with ledgers equal, three times each.
|
||||
3. *"Immich or Nextcloud deploys on 9202"* — Nextcloud did; Immich was not tried.
|
||||
4. **Not in the brief, and the most important:** *"the old `.felhom.yml`"* does not exist at update
|
||||
time. It is replaced on every catalog sync (`09` §5.4). The build now records it when a version is
|
||||
pinned. R-646 for apps pinned before that.
|
||||
**Claims in the brief, checked:**
|
||||
1. *"No app-update event exists today"* — **true for apps.** The only near-misses: `controller_updated` is
|
||||
the controller's OWN self-update, and `update_available` exists only as a debug-page test option — never
|
||||
emitted, and not in the hub's allow-list (it would 400).
|
||||
2. *"`enabled_events` is the household's only mail gate"* — **wrong.** Four more gates stand in the way:
|
||||
the `operatorOnlyEvents` register, a blocked customer, a missing e-mail address, and the cooldown —
|
||||
keyed `customer:type`, which would have merged two apps' mails into one. And the box OVERWRITES
|
||||
the hub's list on each page save (above).
|
||||
3. *"9201 can follow the drill catalog without disturbing its standing apps"* — **true for the catalog,
|
||||
false for the harness.** Measured: the 10 standing apps' compose + `.felhom.yml` were byte-identical
|
||||
before, during and after (the drill differed from live only in the throwaway apps). The disturbance
|
||||
came from the harness's whole-box backup press (R-648), not from the catalog.
|
||||
|
||||
## 2. The bake-off (`audits/undo-bakeoff-2026-09-23/README.md`)
|
||||
## 2. Floor raises, read back from the hub
|
||||
|
||||
Both methods passed every row on docmost, romm, vikunja (seeds A and B back, ledgers equal, cut-off
|
||||
detected before anything moves, ≤ 5.3 s extra downtime). **Folder copy chosen (decision 19):** an app
|
||||
with no database server gets no dump, so dump-and-load would need the folder copy anyway. Rate: 185 MB
|
||||
in 0.81 s; 2 GiB in 4.87 s ≈ 420 MB/s (warm cache); 5 GB ≈ 12 s here, 25–50 s on a cold or spinning
|
||||
disk. Found on the way: the undo must never read the recovery unit (R-645), and killing `docker run`
|
||||
does not stop the copy container.
|
||||
|
||||
## 3. Red-proofs (each seen failing, then restored)
|
||||
|
||||
| # | mutation | test that failed |
|
||||
| raise | read back | fleet |
|
||||
|---|---|---|
|
||||
| 1 | the undo call removed (v0.262.1's shape) | `TestUndo_FailedUpdateIsPutBackWithItsData` — `held=true phase="failed"` |
|
||||
| 2 | copy validation removed | `TestUndo_CutOffCopyIsRefusedBeforeAnythingMoves` — state `half` |
|
||||
| 3 | old-probe rule removed | `TestUndo_UsesTheOldProbe` — `not_started` |
|
||||
| 3b | the old probe read from the stack dir (v0.263.1's shape) | `TestUndo_UsesTheOldProbe` |
|
||||
| 4 | the `undoing` recovery arm removed | `TestUndo_PowerCutDuringTheUndoResumesIt` — `resumed=[]` |
|
||||
| 5a | `last_update_undone` not recorded | `TestUndo_FailedUpdateIsPutBackWithItsData` — `got <nil>` |
|
||||
| 5b | the page line removed from the handler | `TestUndo_PageSaysTheBoxPutTheAppBack` (hu and en) |
|
||||
| 6 | the hold prefix ignored | `TestUndo_HoldSentenceSaysTheUndoWasTriedAndTheDataState` |
|
||||
| 7 | R-642: "completed" back | `TestR642_StartIsNeverReportedCompleted` |
|
||||
| 8 | a successful update keeps the undone note | `TestUndo_ASuccessfulUpdateEndsTheUndoneNote` |
|
||||
| 9 | the undo probe switched off on an `unhealthy` app (v0.263.0's shape) | `TestUndo_OldProbeRunsOnAnAppTheCurrentProbeMarkedUnhealthy` — `not healthy within 1s (last: state unhealthy)`, the live message verbatim |
|
||||
| 10 | the pin advance records no probe | `TestUndo_PinningRecordsThatVersionsProbe` |
|
||||
| 0.263.2 / MinAgent 0.131.0 | `min_controller_version=0.263.2`, `min_agent=0.131.0` | `00-floor-0.263.2.txt` |
|
||||
| **0.264.0 / MinAgent 0.131.0** (14:35:08) | `min_controller_version=0.264.0`, `min_agent=0.131.0`; hub log `managed floor SERVED for demo-felhom: floor 0.264.0 … from declared` | demo-felhom 0.263.2 → **0.264.0 in 20 s**; demo-hp 9201 already 0.264.0 (`50-floor-0.264.0.txt`) |
|
||||
|
||||
`go build ./... && go vet ./... && go test ./...` rc=0 before each of the three commits; controller
|
||||
gates rc=0 (the `docker -v` gate needed four named-volume mounts allowlisted with their why).
|
||||
## 3. The hub migration — rows changed (one-time, add-only)
|
||||
|
||||
## 4. Live proof (`audits/undo-live-2026-09-23/README.md`) — endpoint-level, both languages
|
||||
| customer | before | after |
|
||||
|---|---|---|
|
||||
| `demo-felhom` | 10 types | the same 10 + `app_update_undone`, `app_update_held` |
|
||||
| `demo-hp` | `null` (nothing on) | `["app_update_undone","app_update_held"]` |
|
||||
| `_resend-rotation-test` | `[]` | `["app_update_undone","app_update_held"]` |
|
||||
|
||||
Three apps undone by the product with seeds A, B, C back and ledgers equal (30–52 s of undo); page
|
||||
line hu/en; cut-off copy → HOLD `untouched` (prefix in the box's language, hu and en quoted); power cut
|
||||
during `undoing` → resumed and undone; a person's press after an undo → `done`, note cleared; removal
|
||||
deleted kept copies; R-642 Start answer.
|
||||
Guard row `seed_app_update_events_v1 = 2026-09-23T11:49:03Z`. E-mail column never selected.
|
||||
|
||||
## 5. Rows
|
||||
## 4. The mails (9201, customer demo-hp) and their `notification_log` rows
|
||||
|
||||
**Closed (4, moved to `CLOSED-ITEMS.md`):** R-637, R-639, R-641, R-642. **Narrowed:** R-638, R-640 (to
|
||||
the restore paths). **Ruled:** R-643 (decision 20). **Noted:** R-645. **Opened:** R-645 (earlier this
|
||||
session, filed with the bake-off) and R-646 (apps pinned before v0.263.2). **Open rows 337 → 335.**
|
||||
| row | time (UTC) | type | leg | subject as received |
|
||||
|---|---|---|---|---|
|
||||
| 937 / 938 | 12:15 | undone | op / household | op `⚠️ demo-hp: app_update_undone` · hh **`Figyelmeztetés: vikunja: a frissítés nem sikerült, az alkalmazás a korábbi változattal fut`** |
|
||||
| 939 / 940 | 12:16 | held | op / household | op `🔴 demo-hp: app_update_held` · hh **`Hiba: vikunja: az alkalmazás leállítva, visszaállítás szükséges`** |
|
||||
| 942 / 943 | 12:29 | undone | op / household | hh **`Warning: glance: the update did not work; the app runs on its previous version`** |
|
||||
| 944 / 946 | 12:31 | held | op / household | hh **`Error: glance: the app is stopped and needs a restore`** |
|
||||
|
||||
## 6. Teardown — three layers
|
||||
All 8 `sent`. The box language was switched to English once, between the two rounds (the hub had two
|
||||
reports in English before the second round). Household bodies carry the box's own sentence (the undone
|
||||
line / the hold sentence) in the household's language; the operator's carry the Hungarian wire text.
|
||||
Verbatim bodies: `41-mails-as-received.md`. **Per-app cooldown, live:** row 943 went out 14 minutes after
|
||||
row 938, inside the 6-hour customer cooldown.
|
||||
|
||||
Machine (9202): the three apps and every copy removed through the product; test images removed by
|
||||
name; `controller.yaml` restored and read back; catalog cache on live `cfcfe52`; **9202 stays on
|
||||
controller 0.263.2** (self-update off, fleet floor untouched). Host: nothing provisioned; 9202 stopped
|
||||
and started once for the power cut. Hub: untouched. Drill repo: reset to live `main`.
|
||||
## 5. The pages in both languages (9202, then 9201)
|
||||
|
||||
- undone: hu *„A(z) vikunja frissítése 2026-09-23 14:00-kor nem sikerült. A doboz automatikusan
|
||||
visszaállította az előző változatot és az adatokat — semmi nem veszett el."* · en *"The update of vikunja
|
||||
at 2026-09-23 14:00 did not succeed. The box put back the previous version and its data automatically —
|
||||
nothing was lost."*
|
||||
- held, box hu: `?lang=hu` → the Hungarian hold sentence; `?lang=en` → *"The update did not succeed, and the
|
||||
automatic undo did not either. … own drive, 2026-09-23 13:58 — this copy holds the settings, the database
|
||||
and the data volumes."* — the whole sentence English now (before v0.264.0 only the prefix was).
|
||||
- `GET /api/stacks/vikunja?lang=en`: label *"The update did not succeed"*, error and hold in English.
|
||||
- **Leftover (R-647):** box en + `?lang=hu` showed the English hold.
|
||||
|
||||
## 6. Red-proofs (each seen failing, then restored)
|
||||
|
||||
**Controller (9):** backfill stores nothing → `TestR646_…` *"the current app must have its record"*;
|
||||
`dropped()` not called → `TestR620_…` *"want exactly TWO WARN lines, got 0"*; undone event not emitted →
|
||||
*"exactly ONE app_update_undone event, got []"*; held event not emitted → *"…got []"*; page
|
||||
`updateErrorText` not localised → `TestR606_UpdateSentencesFollowTheReader`; hold sentence ignores the
|
||||
language → `undo_hold_test.go:62`; `UpdateErrorIn` returns the Hungarian → *"English = „A frissítés nem
|
||||
indult el…""*; box seed not run → *"got [backup_failed offbox_enlarge_blocked]"*; sink not wired in main →
|
||||
*"SetUpdateEventSink is never called"* (first mutation did not compile; retried with a compiling one).
|
||||
**Hub (5):** seed call does nothing → *"c1: enabled_events = [backup_failed], want […]"*; customer per-app
|
||||
suffix removed → *"want 3 household mails …, got 2"*; allow-list entry removed → *"must be in
|
||||
allowedEventTypes"*; app-named headline removed → subject *"%s: a frissítés…"*; seed default removed →
|
||||
*"defaultSeedEvents must include app_update_undone"* (first attempt matched no test; retried with a new
|
||||
test that names both types). Every `-run` was checked for `=== RUN`.
|
||||
|
||||
Gates: controller `go test ./...` rc=0, `controller_gates.py` rc=0; hub `go build/vet/test` rc=0,
|
||||
`repo_gates.py` rc=0. The R-389 fence test was widened on purpose, with its reason; the settings-page
|
||||
parity fixture was regenerated (measured diff: exactly the two new toggles).
|
||||
|
||||
## 7. Rows
|
||||
|
||||
**Closed (3, moved to `CLOSED-ITEMS.md`):** R-606, R-620, R-646. **Opened (2):** R-647 (the three
|
||||
leftovers), R-648 (the whole-box backup press). **Open rows 335 → 334.** `09` §6.4 parts 2 and 3 marked
|
||||
SHIPPED; the capability map's undo row now carries the mail; STATUS rewritten.
|
||||
|
||||
## 8. Teardown — three layers
|
||||
|
||||
- **machine:** 9202 and 9201 — throwaway apps removed through the product (0 volumes, 0 undo copies),
|
||||
test images removed, `controller.yaml` identical to the saved copy, catalog on live `cfcfe52`, 9201's
|
||||
language back to `hu`, standing apps deployed and healthy. Both on 0.264.0 (= the floor).
|
||||
- **host:** nothing provisioned on demo-hp.
|
||||
- **hub:** v0.120.0 (planned); floor 0.264.0. Scratchpad password files deleted; hub DB copies deleted.
|
||||
- **drill repo:** reset to live `main`, read back from the remote.
|
||||
|
||||
Reference in New Issue
Block a user