admin/felhom-controller
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v0.153.0 — R-47: the DB replay no longer races the app, on BOTH restore paths

Browse Source 
Closes R-47. No new agent coupling — MinAgent stays 0.90.0.

The replay needs a running DB container, so both restore paths started the
WHOLE stack first, giving the application a window to rebuild the very schema
objects the dump was about to create. Measured live on 2026-07-19 (H4,
DIAG-immich-restore-round2): immich-server rebuilt clip_index two seconds
before the dump's CREATE INDEX, the replay aborted "already exists" under
ON_ERROR_STOP=1, and immich reported schema drift. The data survived only
because pg_dump emits COPY before CREATE INDEX.

Both paths now open a DB-ONLY window: only the stack's database service(s)
come up, the dump is replayed with the app still down, and the full start
runs only after the replay exits 0. Fail-closed: a dump with no identifiable
DB service refuses BEFORE the first mutation. Every exit from the window
still does a best-effort full start, so a failed restore never leaves a box
with a database and no application.

New: appbackup.DBServiceNames (yaml.v3 services-map parse — never a line
scan; immich's top-level volume keys are the decoy) sharing dbTypeForImage
with DiscoverDatabases; stacks.Manager.StartStackServices (refuses an empty
list — argument-less `up -d` is a full start); RedeployFromEnv split into
PersistUnitRedeployConfig + its unchanged tail. StackDataProvider's
RecreateStackFromUnit becomes RecreateStackDefinitionFromUnit — the hidden
`up -d` inside the old name is what carried the defect on the local path.

19 new tests (ordering plus state-at-replay-time, zero-mutation fail-closed
effects, replay-failure bring-up, parser decoys, empty-list refusal); three
companion red-proofs run and reverted. 23/23 packages green.

Not yet live-validated: STOP-1 supervised reconstitute, golden 0.153.0.
This commit is contained in:
authentik Default Admin
2026-07-20 17:01:52 +02:00
parent fd40b29119
commit 78ff991f1c
25 changed files with 1323 additions and 201 deletions
Download Patch File Download Diff File Expand all files Collapse all files
REPORT.md
+103 -123
View File
@@ -1,153 +1,133 @@
# REPORT — v0.152.0 + felhom-samba 1.1.0: mDNS for macOS (S-3), and the connect card stops offering a dead form
# REPORT — R-47: the DB replay must not race the app (both restore paths) · felhom-controller v0.153.0
**Date:** 2026-07-20 · **Repo:** felhom-controller (v0.151.0 → **v0.152.0**) + **felhom-samba
1.0.0 → 1.1.0** · Trunk, pushed to `main`.
**Origin:** operator report — `smb://FELHOM` still failing from a Mac after v0.151.0; S-3 of
`felhom.eu/documentation/audits/DIAG-sharing-2026-07-20.md`.
**Date:** 2026-07-20 · **Repo:** `felhom-controller` (v0.152.0 → **v0.153.0**) · Trunk, pushed to
`main`. · **Baseline:** `main` @ `fd40b29` (clean, equal to `origin/main` at session start)
## Baselines
---
| Repo / artifact | start | end |
|---|---|---|
| felhom-controller | `5c105fb`, v0.151.0 live on 9201 | `37e12c8` (+ this docs commit), **v0.152.0** |
| felhom-samba | `1.0.0` | **`1.1.0`**, digest `sha256:1c17c094…` |
| felhom.eu | `a7d9837` | `1d1d60a` |
## 1. What was wrong
## The finding that redirected the fix
My earlier conclusion — "macOS no longer does NetBIOS" — **was wrong**, and a packet capture on the
box disproved it. On a bare `smb://FELHOM` from the operator's Mac (`192.168.0.11`):
`felhom.eu/documentation/audits/DIAG-immich-restore-round2-2026-07-19.md`, finding **H4**. The
offsite reconstitution ran its designed sequence — safety dump → stop → start → replay — and the
replay aborted:
```
11:18:30.222080 .11:52844 > 192.168.0.255:137 NBNS query, 50 B
name → "FELHOM" + 9 pad + suffix 0x20 (File Server Service — correct for SMB)
11:18:30.222220 .104:137 > .11:52844 NBNS response, 62 B, +140 µs
flags 0x8580 = response, AUTHORITATIVE, RCODE=0 · ANCOUNT 1
TTL 259200 · NB_FLAGS 0x0000 (unique, B-node) · RDATA 192.168.0.104
10:58:25 controller: replaying DB dump into immich-postgres
10:58:33 immich-server: "Reindexing clip_index" -> "Reindexed clip_index"
10:58:35 controller: ERROR relation "clip_index" already exists - exit status 3
```
**macOS asked correctly, was answered correctly in 140 microseconds, and never opened a TCP
connection.** It retried once, was answered again, gave up. Sixteen seconds later the same Mac
queried `FELHOM.local` over mDNS and went straight to port 445.
`ImportDump` needs a running database container, so the code started the WHOLE stack first. That gave
the application an eight-second window to rebuild the very schema objects the dump was about to
create; under `ON_ERROR_STOP=1` the collision aborted the script. The data survived only because
`pg_dump` emits COPY before CREATE INDEX — a collision earlier in the script would have left a
genuinely half-restored database and reported it identically.
So NetBIOS on macOS feeds legacy browsing, not `smb://` URL resolution: **the bare `smb://<NÉV>`
cannot be made to work from a Mac by any change on our side**, and nmbd — which the R-6/S4b spike
was right to insist on — was never the broken part. It is precisely what serves Windows.
**Class defect.** The local `RestoreFromRecoveryUnit` had the same start-then-replay shape, hidden
inside `RecreateStackFromUnit` (which ended in a full `compose up -d`). Both are fixed here.
Investigated and dismissed: the box answers **twice** per broadcast (nmbd holds `0.0.0.0:137`,
`<ip>:137` and `<bcast>:137`; a broadcast lands on two). Standard Samba, and a duplicated correct
answer is still a correct answer that macOS declined to use.
## 2. What was built
### Proven matrix
**Part 1 — the seams**
| Client | Working form | Served by |
|---|---|---|
| Windows | `\\<NÉV>` | nmbd (+ wsdd for the Network view) |
| macOS | **`smb://<NÉV>.local`** | avahi/mDNS — new in 1.1.0 |
| Any | `smb://<IP>` | direct |
## Method — spike before publish (operator's call, and it paid)
avahi was installed by hand into the *running* container and configured, then proven from a second
machine before any image was built. The operator confirmed `smb://FELHOM.local` connects. Only then
was 1.1.0 built. The spike also chose the design: a **static avahi service file** beats smbd's own
`multicast dns register`, because `smb.conf` is bind-mounted READ-ONLY and owned by the controller's
renderer, and a static file additionally publishes `_device-info._tcp`.
## Changes
| File | Change |
| Change | File |
|---|---|
| `controller/infra-images/samba/Dockerfile` | `avahi` + `dbus` added; `rm` of packaged service files; header documents the captured NBNS proof and that sidebar discovery is NOT claimed |
| `controller/infra-images/samba/entrypoint.sh` | templates `avahi-daemon.conf` + `_smb._tcp`/`_device-info._tcp` service file **from `FELHOM_SERVER_NAME`**; starts dbus + avahi, both **non-fatal** |
| `controller/internal/infra/infra.go` | `SambaImage` pin → `1.1.0` (`Images()` and the golden bake follow automatically) |
| `controller/internal/web/templates/sharing.html` | Mac line `smb://<NÉV>` → **`smb://<NÉV>.local`**; Windows flat name untouched |
| `controller/internal/web/sharing_connect_card_test.go` | assertions updated + new `TestSharingConnectCard_MacLineIsDotLocalNotBareName` |
| `controller/internal/infra/samba_test.go` | tag assertion derives from `SambaImage`; adds an explicit non-`:latest` assertion |
| `controller/internal/web/handler_export_upload_test.go` | async-race fix (below) |
| `CHANGELOG.md`, `controller/README.md`, `REPORT.md` | docs |
| `dbTypeForImage` extracted from `DiscoverDatabases` (behaviour byte-equivalent) and shared | `internal/appbackup/dbdump.go`, `internal/appbackup/dbservices.go` (new) |
| `DBServiceNames(composePath)` — sorted compose SERVICE names holding a DB; yaml.v3 `services:` map parse | `internal/appbackup/dbservices.go` (new) |
| `Manager.StartStackServices(name, services)` — scoped `up -d`, **refuses an empty list** | `internal/stacks/manager.go` |
| `RedeployFromEnv` split; persist half is `PersistUnitRedeployConfig` (starts nothing) | `internal/stacks/deploy.go` |
| `StackDataProvider`: `RecreateStackFromUnit` → `RecreateStackDefinitionFromUnit` (+ `StartStackServices`) | `internal/appbackup/appdata.go` |
| Adapter: definition-only recreate + delegation | `cmd/controller/main.go` |
| `DBServiceNames` forwarder | `internal/backup/appbackup_bridge.go` |
## Tests
**Part 2 — offsite** (`internal/backup/offbox_reconstitute.go`): DB services resolved from the LIVE
compose before any mutation; fail-closed refusal when a DB exists but no service is identifiable;
sequence is now **stop → files → `StartStackServices(dbServices)` → replay → `StartStack` (full) →
health wait**; both failure exits from the window do a best-effort full start.
**23/23 packages green, run twice with `-count=1`.**
**Part 3 — local** (`internal/backup/restore_unit.go`): DB services resolved from the UNIT's compose
(it is about to become the live one) plus `hasReplayableDump` (excludes `pre-restore-` safety dumps);
same fail-closed gate before the first mutation; sequence is now **stop → volumes →
`RecreateStackDefinitionFromUnit` → `StartStackServices` → replay → `StartStack` (full) → health
wait**, with the pre-existing `dataErr` / "completed with data errors" semantics preserved.
Red-proofs, both mutated → FAILED → restored:
Untouched, as specified: `restore_db.go`, `ImportDump`, `waitDBReady`, the dump flags
(`--clean --if-exists`, `ON_ERROR_STOP=1`), `mapOffsiteRestorePaths`, the copiers, the honesty
surfaces, `IsDownState`/alerting (R-51), and the agent/hub.
| Mutation | Result |
|---|---|
| revert the template's Mac line to the bare `smb://{{.SMBServerName}}` | `TestSharingConnectCard_MacLineIsDotLocalNotBareName` **FAILED** on both the missing `.local` and the present bare form, for both configured names (`FELHOM`, `OTTHON`) |
| delete `os.Remove(job.partPath)` from `expireIdleUpload` | `TestFabUpload_GCAndIdleTimeout` **FAILED** — `idle-expired .part must be deleted` |
## 3. Tests — 19 new, Groups A–G
### Two test bugs found, neither a production defect
| Group | Test | Result |
|---|---|---|
| A | `TestReconstituteReplaysWithOnlyTheDBServiceUp` — order **plus state-at-replay-time** | PASS |
| A | `TestReconstituteReplaysDBAndOrdersOperations` (existing, sequence assertion updated) | PASS |
| B | `TestReconstituteNoDBAppNeverStartsServicesOnly` — negative, zero scoped starts | PASS |
| C | `TestReconstituteRefusesWhenNoDBServiceIdentifiable` — zero-mutation effect | PASS |
| C | `TestRestoreFromUnitRefusesWhenNoDBServiceIdentifiable` — zero-mutation effect | PASS |
| D | `TestRestoreFromUnitReplaysWithOnlyTheDBServiceUp` | PASS |
| D | `TestRestoreFromUnitNoDumpsTakesOneFullStart` | PASS |
| D | `TestRestoreFromUnitIgnoresSafetyDumpsWhenDecidingToReplay` | PASS |
| E | `TestReconstituteReplayFailureStillBringsTheStackUp` | PASS |
| E | `TestReconstituteDBOnlyStartFailureStillBringsTheStackUp` | PASS |
| E | `TestRestoreFromUnitReplayFailureStillBringsTheStackUp` | PASS |
| F | `TestDBTypeForImage`, `TestDBServiceNames` (8 sub-cases), `TestDBServiceNames_TopLevelKeysAreNotServices`, `TestDBServiceNames_UnreadableAndUnparseableError`, `TestDiscoverAndComposeAgreeOnTheSameImages` | PASS |
| G | `TestStartStackServicesRefusesEmptyList`, `TestPersistUnitRedeployConfigPersistsWithoutStarting`, `TestPersistUnitRedeployConfigRejectsUnknownStack` | PASS |
1. **`TestRenderSambaCompose` pinned the literal tag `1.0.0`**, so a routine image bump read as a
renderer regression. Now derives from `SambaImage`, plus a separate assertion for what actually
matters — the tag is explicit and never `:latest`.
2. **`TestFabUpload_GCAndIdleTimeout` raced.** `expireIdleUpload` nils the slot, releases the mutex,
and only *then* closes and unlinks the `.part` — so "the slot is free" does not yet mean "the file
is gone", and the test stat-ed immediately. It passed in isolation and failed in the full package
once this release's new render tests made `web` heavier. **Not caused by this change and not a
production bug** (a new upload mints a fresh random `.part`, so the gap is harmless); the test was
asserting an async post-condition synchronously. It now waits on the same 3 s deadline with the
assertion unchanged.
The core assertion is deliberately not "no error": a recording provider captures whether the FULL
stack had been started at the moment the import fired. Asserting only `err == nil` passes on the
pre-fix shape — which is exactly how this shipped.
Design gates `template_id_gate` / `emoji_gate` / `native_confirm_gate` / `offbox_rename_gate`: **OK**.
The compose-parser decoys use the catalog's REAL immich template shape (`immich_ml_cache:`,
`immich_postgres_data:` as top-level `volumes:` keys, `ghcr.io/immich-app/postgres:16-vectorchord…`
as the pin) — the exact input a line scan would misread.
## Build, publish, deploy
### Companion red-proofs — three run, all reverted, tree clean
```
./controller/scripts/build-samba-image.sh 1.1.0 --push → sha256:1c17c09422be…
smoke test (DooPlex, DEFAULT BRIDGE — never --network host on this host, it would bind 445/5353)
→ smbd · nmbd · wsdd · dbus-daemon · avahi-daemon: running [SMOKETEST.local] (all 5 up)
→ host-name=SMOKETEST, allow-interfaces=eth0, _smb._tcp/445 in the service file
anonymous pull from guest 9201 (no registry creds) → OK
./build.sh 0.152.0 --push → deployed to 9201
```
| # | Pre-fix shape restored | Failure observed |
|---|---|---|
| 1 | offsite: `StartStackServices` → full `StartStack` before the replay | `TestReconstituteReplaysWithOnlyTheDBServiceUp`: *"the database service was NOT started before the replay"*; `TestReconstituteReplaysDBAndOrdersOperations`: sequence `"stop,start,start"` |
| 2 | local: full `StartStack` inserted before the replay | `TestRestoreFromUnitReplaysWithOnlyTheDBServiceUp`: *"the FULL stack was already up when the replay fired — the H4 race, on the local path"* |
| 3 | both fail-closed gates deleted | both `RefusesWhenNoDBServiceIdentifiable` tests: *"expected a refusal…"* |
Live: `felhom-controller:0.152.0 Up (healthy)` · `felhom-samba:1.1.0 Up` — the controller's reconcile
recreated the samba container onto the real image, discarding the hand-patched spike.
### Green gate
## Live validation
`go build ./... && go vet ./... && go test ./...` — **23/23 packages green**, exit 0
(`internal/backup` 174 s). New tests by package: backup +11, appbackup +5, stacks +3.
**Method: on-the-wire from a second machine (felhom-pve) + endpoint-level for the page.**
## 4. Deployment
```
mDNS from felhom-pve, against the SHIPPED image:
FELHOM.local (A) -> A 192.168.0.104 (from 192.168.0.104)
_smb._tcp (browse) -> PTR FELHOM._smb._tcp.local (from 192.168.0.104)
Recorded on completion of Phase B — see the CHANGELOG entry for the shipped version.
daemons in felhom-samba:1.1.0: smbd · nmbd · wsdd · dbus-daemon · avahi-daemon: running [FELHOM.local]
## 5. NOT yet live-validated — remaining human/supervised work
GET /sharing -> smb://FELHOM.local smb://192.168.0.104 \\FELHOM
GET /sharing/status ×2 -> phase:"idle", running:true (v0.151.0 contract still holding)
```
- **STOP-1 (supervised, Viktor present):** prepare a full offsite restore scratch for immich through
the real endpoints, then fire `/backup/offbox/reconstitute` and verify through the system's own
surfaces — controller log showing stop → db-only up → replay rc-0 → full up, no `already exists`,
no drift, immich healthy with content visible. Timestamps to be recorded here afterwards.
- **Phase C (golden 0.153.0):** probes P1–P3 first. **P3 is load-bearing** — the drill environment is
at the vacation site and must be proven able to pull
`gitea.dooplex.hu/admin/felhom-controller:0.153.0` BEFORE any bake. If unreachable: stop and
report; change no routing/DNS/nft.
- **STOP-2 (Viktor, password-gated):** Day-0 manifest Golden → 0.153.0 (Agent 0.90.1 / MinAgent
0.90.0 unchanged — the CHANGELOG's no-coupling declaration is the authority), then floor →
v0.153.0 saved **LAST**. Watching the demo box wake during the manifest save banks the **R-23(a)**
operator-UI save→apply evidence — log the timestamps if observed.
- **Viktor's C6 customer-restore UI run** (note the empty-the-trash method).
Operator, from the Mac: **`smb://FELHOM.local` connects and prompts for credentials.**
## 6. Observations (out of scope, recorded not acted on)
## Still open
- **Finder-sidebar discovery — NOT shipped, NOT claimed.** The `_smb._tcp` record is published and
answers browse queries on the wire, but FELHOM did not appear in the operator's Finder sidebar.
That window showed no Network/Bonjour section at all, which points at **Finder Settings → Sidebar
→ Locations** rather than at the box — unverified either way. Next: that setting, and
`dns-sd -B _smb._tcp` on the Mac. Zero-typing discovery was the R-6 spike's original ambition and
is still not demonstrated.
- **Windows was not retested** this session. `\\FELHOM` is served by nmbd, which this release does
not touch, and the capture proves nmbd answers correctly — but no Windows client was exercised.
## Observations — noticed, not acted on
- **The username trap.** macOS prefills the local account name (`Viktor.Nagyfenyvesi`) in the SMB
credential dialog; the household account is `felhom`. The page states this in the password card,
but not next to the connect addresses where the customer is looking at that moment. Worth a
sentence in the connect card — deliberately not added mid-session without a design pass.
- **nmbd's duplicate answer** is harmless here but would look alarming in any future capture. Noted
in the DIAG so the next person does not re-investigate it.
- **The spike left avahi installed by hand** in the old container. It was discarded when the
reconcile recreated the container on 1.1.0 — worth remembering that hand-patching a *managed*
infra container is always temporary by construction, which is a safety property, not a limitation.
- **`dns-sd`/`avahi-browse` were not available** anywhere on the box side, so both the NBNS and mDNS
verifications were done with small hand-written UDP probes run from felhom-pve. That turned out to
be the more valuable method anyway: querying from a *second machine* is what distinguishes "the
daemon answers itself" from "the daemon answers the network", and it is what proved the box
innocent before any code changed.
- **The `pre-restore-` prefix is load-bearing in three separate places** (the replay's exact-name
match, `OffsiteScratchPair`'s dump sniff, and now `hasReplayableDump`) with no shared predicate
deciding "is this file a replay source". A fourth consumer that forgets the exclusion would arm the
DB-only window for an app that has nothing to replay. Worth a single helper at some point.
- **`reimportDBDumpsFrom`'s own `hasDump` scan does NOT exclude the safety prefix** (unchanged here —
`restore_db.go` was explicitly out of scope). Harmless today because the per-DB lookup is an exact
`<stack>-<dbtype>.sql` match, so a directory holding only safety dumps merely produces the
"no matching running DB container" WARN instead of a clean zero.
- **`dbTypeForImage` maps both `mysql` and `mariadb` to `DBTypeMariaDB`.** Pre-existing and correct
for the current catalog (the `mariadb` client speaks to both), but it is an assumption, not an
invariant, and it is now written down in one place instead of two.
- **`RedeployFromEnv` has no end-to-end test** (it shells out to compose), so the split's equivalence
is asserted on the persist half only. That is the half the split could break; the tail is
byte-identical code that was moved, not rewritten.
- **R-29a (`estimate.go` gate finding)** remains open and was not touched.