docs(shares): REPORT.md — R-7b v0.145.0 ship report

Findings, all six red-proof outcomes, deployment, live-validation evidence, and an
explicit list of what was NOT live-exercised (offsite leg + restore round-trip: the
demo box has no offsite target).
This commit is contained in:
2026-07-18 13:34:59 +02:00
parent d0c1d77491
commit a5d870ea0e
+146 -148
View File
@@ -1,182 +1,180 @@
# REPORT — most recent implementation
## felhom-controller v0.144.0 — „Megosztás": LAN SMB file sharing (R-7 slice 1) — 2026-07-18
## felhom-controller v0.145.0 — R-7b: share data enters the live backup runs (Model B) + samba liveness — 2026-07-18
**Class:** Implementation. **Shipped:** controller **v0.144.0** + new infra image
**`gitea.dooplex.hu/admin/felhom-samba:1.0.0`**. Deployed + live-validated on demo guest 9201.
**Class:** Implementation. **Shipped:** controller **v0.145.0**, deployed and live-validated on demo
guest 9201.
### 1. Baselines
## 1. Baselines (re-confirmed at session start)
| Repo | start `main` @ | Version |
| Repo | `main` @ start | Version | → Shipped |
|---|---|---|---|
| felhom-controller | `3dfc49e` | v0.144.0 | **v0.145.0** |
| felhom.eu | `b76dad0` | docs tip | docs only |
Trunk-based, explicit-path staging, no Co-Authored-By.
> The repo root also held a stale, unrelated `TASK.md` ("Telemetry Debug Section"). It was ignored —
> the R-7b prompt was the spec.
## 2. What shipped, and why it matters
Before this release the „Megosztás" page rendered „Felhőmentés: bekapcsolva" for a share whose files
were **in no backup at all**. `backup.RunTier2` short-circuits on `os.Stat(unitDir)` before the
classification seam, and the offsite runner enumerates `GetOffboxApps()`; a share-only infra stack has
neither a recovery unit nor an offbox toggle, so it fell through both engines. The toggle is now true.
**Model B (Viktor's ruling) and its invariant.** Share data enters the runs through a **sibling
shares source** — additive job/leg code reusing the proven primitives while leaving **every per-app
engine path byte-identical**. That invariant is the headline claim and is enforced by test in both
tiers, each with a red-proof.
| Part | File | Shape |
|---|---|---|
| felhom-controller | `ac13966` (v0.143.0) | → **v0.144.0** |
| felhom.eu | `7e0370f` | docs only (hub v0.66.0 / scripts v1.20.0 untouched) |
| felhom-agent | `f222a7b` | NOT touched (v0.90.0) |
| app-catalog-felhom.eu | — | NOT touched (deliberately not a catalog app) |
| 1 payload | `internal/backup/shares_payload.go` | deterministic `_shares-manifest.json` + best-effort `passdb.tar` |
| 2 tier-2 | `internal/backup/tier2_shares.go` | per-source-drive legs → `backups/secondary/_shares/<driveKey>/<share>`, `_payload/`, marker LAST |
| 3 offsite | `internal/backup/offbox_shares.go` | ONE `restic backup --tag felhom-offbox --tag _shares`, after the app loop, before retention |
| 4 restore | `internal/backup/shares_restore.go` | scratch → prefix-asserted missing-only merge → definitions (existing-wins) → `ReconcileSamba` → credential |
| 5 liveness | `internal/monitor/healthcheck.go` | `EffectiveProtected` gains a settings-backed dynamic extra |
| 6 UI | `sharing.html`, `backups_restore.html`, handlers | per-tier status lines; „Megosztások" restore entry |
*(The brief listed `a4a7de3d` for the controller; the live baseline was `ac13966` — the delta is the
2026-07-18 build-root relocation, paths/docs only.)*
**Seams extracted (never forked):** `selectTier2TargetFrom` (source drive supplied explicitly;
`selectTier2Target` is now a thin wrapper — the headroom math is untouched) and `tier2ReconcileRoots`
(pure extraction; `tier2Reconcile` calls it with `hdd`/`userdata`).
### 2. Files created / modified
**Degradation contract:** a quota-blocked offsite push falls back to the **manifest only, never to
nothing** — definitions protection must not regress because the files stopped fitting.
**Created:** `controller/infra-images/samba/{Dockerfile,entrypoint.sh}`,
`controller/scripts/build-samba-image.sh`, `internal/settings/smb.go` (+test),
`internal/infra/samba.go` (+test), `internal/stacks/samba.go` (+test),
`internal/stacks/samba_classify.go` (+test), `internal/web/sharing_handlers.go` (+test),
`internal/web/templates/sharing.html`.
## 3. Commits
**Modified:** `settings/settings.go` (2 fields), `infra/infra.go` (`SambaImage` pin),
`stacks/infra.go`, `stacks/manager.go` (3 test seams), `stacks/metadata.go` (samba branch),
`config/config.go` (`alwaysProtectedStacks`), `web/server.go`, `web/inframeta.go`,
`cmd/controller/main.go` (`/api/sharing/` mux), `templates/layout.html`, `templates/icons.html`,
`CHANGELOG.md`.
### 3. Commits
| Hash | Phase |
| Hash | Subject |
|---|---|
| `f42f3e0` | Part 0 — felhom-samba image + build helper |
| `b0c5ef4` | Part 1settings registry + renderers |
| `0dcbea9` | Part 2 — lifecycle (ensure/reconcile/password/disable) + protection |
| `1d26a69` | Part 4 — backup classification from the shares registry |
| `4f08e5e` | Part 3 — „Megosztás" page + guarded picker |
| `2eef9b2` | fix: `/api/sharing/` mux registration (found by live validation) |
| `b409f5e` | fix: storage-ROOT vs share-TARGET validation split (found by live validation) |
| `c81df55` | Parts 12 — payload builder + tier-2 shares job |
| `85b76e0` | Part 3offsite shares leg + B isolation proof |
| `900c870` | Parts 46 — restore, samba liveness, UI truth-up |
| `3b70a9e` | docs — CHANGELOG/CONTEXT/REUSE/README; caveats cleared |
| `d0c1d77` | fix — reserved key leaked into the `crossdrive_completed` hub event (found live) |
### 4. Part-4 Step-1 enumeration finding (verbatim, as required)
felhom.eu: capability-map SMB row, `backlog/ROADMAP.md` R-7b, `controller/sharing.md`.
`GetStackClassifiedBinds` is **NOT inert** — it is consumed by `backup/offbox_capture.go:36`
(offsite), `backup/tier2_capture.go:45` (tier-2), `appexport/fabplan.go:33,115` (.fab). The
enumeration nevertheless never reaches a share-only stack:
## 4. Findings
- **Tier-2** — `RunAllTier2` (`tier2.go:399`) iterates `ListDeployedStacks()` and does **not** skip
protected stacks. But `RunTier2` (`tier2.go:269`) resolves `GetAppDrivePath(stack)` then
`os.Stat(RecoveryUnitPath(...))` and **returns nil before `tier2CaptureSet` is ever called** when no
recovery unit exists. samba has neither an app drive path nor a recovery unit.
- **Offsite** — `offbox.go:600` enumerates `settings.GetOffboxApps()` (apps with
`AppBackup[name].Offbox`); samba is not an app-backup app.
- **Volume dumps** — `backup.go:437439` iterates `ListDeployedStacks()` and `continue`s on
`cfg.IsProtectedStack(name)`; with samba protected this correctly skips it.
**(a) The reserved-name assumption in the task was FALSE — guard added.** The task asked me to verify
at source that slice-1 validation excludes a leading underscore. It does **not**: `settings.nbNameRe`
begins with `[A-Za-z0-9_]`, so „_shares" was an **accepted share name**. `ValidateSMBShareName` now
refuses the leading-underscore namespace (on ADD only, so existing shares are never retroactively
invalidated). Stack names come from the git-synced catalog rather than customer input, so a `_shares`
STACK is not realistically reachable — but `RunAllTier2` and `RunOffboxBackup` skip one loudly as
defense in depth rather than let it clobber the shares tree.
**Verdict — design fork STOPPED and reported, not improvised** (per the task's Part-4 clause): putting
share data into a live tier-2/offsite run requires teaching both engines about a stack with **no
recovery unit and no single app drive path** — a structural change inside the engines, far beyond an
enumeration tweak. Slice 1 therefore delivers the **correct classification seam** only.
**(b) Part 5 — the alert pipeline needed no further change.** A missing protected container →
`report.Issues``Status="fail"``notifier.NotifyHealthChange` → the **existing** `health_critical`
event. No new event type is introduced, so the `allowedEventTypes` gotcha does not apply. Verified
live: the hub accepted the event with HTTP 200.
**Consequence, stated plainly:** [R4]'s intent — a customer dropping family files onto `\\FELHOM` gets
the product backup promiseis **classified but not executed**. Share data is in no live backup run
today. Needs a Viktor ruling (suggested new item **R-7b**).
**(c) A real bug, surfaced by test.** `shareSourceDrive` returned a slash-normalised path, which made
the target selector's source-drive equality check missa share group could have targeted **its own
source drive**, i.e. a same-disk copy pretending to be tier 2. POSIX-only in effect (`ToSlash` is
identity there), but a genuine defect. Fixed.
### 5. Tests
**(d) A real leak, surfaced by LIVE VALIDATION, not by any unit test.** The first demo run pushed a
hub event reading „Másodlagos mentés elkészült: **_shares**" — the reserved key reached Hungarian
customer/operator copy through the `tier2Notify``crossdrive_completed` path I had not mapped. Fixed
at the **source** of the notification (so no future notifier wiring can reintroduce it), with
`DisplayStackName` at the main.go wiring as idempotent defense in depth, plus a regression test and a
red-proof. Re-validated live: the event now reads „…: **Megosztások**". This is the clearest argument
for the live leg — six green red-proofs did not catch it.
Test functions **651 → 669** (+18). Green gate (`go build ./... && go vet ./... && go test ./...`) run
after every phase: **23/23 packages ok**, zero failures. Template gates green
(`template_id_gate.py`, `emoji_gate.py`).
**(e) Two warning-prose sites need no mapping.** `offbox_capture.go` and `tier2_capture.go` embed a
raw stack name in Hungarian prose, but the shares source is a sibling and never flows through the
capture-set helpers, so `_shares` cannot reach them. Verified rather than assumed; no edit made.
| Scenario | Tests | Result |
**(f) Pre-existing gate failure, untouched.** `scripts/docker_run_volume_path_gate.py` fails on
`internal/appexport/estimate.go:179`. It fails identically on the unmodified tree (verified by
stashing), predates this work, and is out of scope. All six other gates pass.
## 5. Tests
`go build ./... && go test ./...`**green** (23 packages). 25 new/extended cases in
`internal/backup` + 2 in `internal/monitor`.
### Red-proofs — all six run, all fired, all reverted
| # | Break introduced | Test that failed |
|---|---|---|
| A — enable + first share | `TestSambaReconcile_HappyPath`, `TestRenderSambaConfig_Golden`, settings CRUD | PASS |
| B — read-only share | `TestRenderSambaCompose` (`read only = yes` + `:ro` bind) | PASS |
| C — security gates | `TestSharingResolvePath_{GuardMatrix,SymlinkEscapeRefused,DecommissionedRootRefused,UniformRefusal}`, `TestSharingResolveStorageRoot`, `TestPathWithin_*`, name validation | PASS (symlink case SKIPs on Windows — no privilege) |
| D — classification | `TestSambaClassifiedBinds_{TierMembership,ExcludesConfigMounts,NoShares}` | PASS |
| E — delete/disable keep data | `TestSambaShareDeleteAndDisableKeepData` (byte-identical tree) | PASS |
| Idempotency | `TestSambaReconcile_IdempotentNoComposeCall` (seam asserts 0 compose calls) | PASS |
| Secrets | `TestSambaPasswordNeverPersisted`, `TestSambaPasswordRefusedWhenDisabled` | PASS |
| 1 | shares leg appends its paths into the **app's** argv | `TestOffboxSharesLegLeavesAppCallsByteIdentical` — "B INVARIANT VIOLATED" |
| 2 | mandatory→offsite mapping inverted | `TestOffboxSharesLegPushesMandatoryShare` + `…ExcludesOptionalShare` |
| 3 | manifest-only degradation dropped (skip leg when blocked) | `TestOffboxSharesLegQuotaDegradesToManifestOnly` |
| 4 | prefix-assert removed (`liveShareRootOK` → true) | `TestSharesRestoreRefusesDestinationOutsideLiveRoots` — "PLACE GUARD BREACHED" ×2 |
| 5 | dynamic samba extra removed | `TestEffectiveProtectedTracksSharingToggle` (enabled case) |
| 6 | shares `destBase` drops the reserved segment | `TestSharesTier2LeavesPerAppTreeUntouched` |
#### Red-proofs — all four run, confirmed, restored (`git diff` clean each time)
Plus the post-hoc regression: passing the raw key to `tier2Notify` fails
`TestSharesTier2NotifierNeverLeaksReservedKey`.
1. **Classification** — inverted Offsite→mandatory → `TestSambaClassifiedBinds_TierMembership` FAILED:
`Felhőmentés ON must be mandatory, got "optional"` + `mandatory share must be in the OFFSITE set: []`.
2. **`:ro` bind** — dropped the read-only bind mapping → `TestRenderSambaCompose` FAILED:
`read-only share must get a :ro bind, missing "- …/filmek:…/filmek:ro"`.
3. **Picker guard** — removed the containment assert in `sharingOwningRoot`
`TestSharingResolvePath_GuardMatrix` FAILED:
`outside every registered root: must be refused, got "…\Temp\…\002"`.
4. **Idempotency** — removed config change-detection → `TestSambaReconcile_IdempotentNoComposeCall`
FAILED: `unchanged registry must perform NO compose call: calls went 1 → 2`.
#### Password-leak check (rule 4)
`TestSambaPasswordNeverPersisted` asserts the secret reaches the smbpasswd seam and appears in **none**
of `settings.json`, `smb.conf`, `docker-compose.yml`. Live: the demo `settings.json` carries only
`"user_set": true`. The password is STDIN-only (never argv) and never logged.
### 6. Deployment (live)
## 6. Deployment
```
felhom-controller: gitea.dooplex.hu/admin/felhom-controller:0.144.0 Up (healthy)
felhom-samba: gitea.dooplex.hu/admin/felhom-samba:1.0.0 Up
gitea.dooplex.hu/admin/felhom-controller:0.145.0 Up (healthy) guest 9201
```
Anonymous pull of the samba image verified from guest 9201 without registry creds (Part-0 gate).
Built on 180 (`/mnt/5_hdd/felhom.eu/build/felhom-controller`, explicit pull first), deployed via the
bootstrap service. Deployed twice — the second time carrying the finding-(d) fix.
Binds (`docker inspect felhom-samba`) — Scenario B proof:
## 7. Live validation (demo, through the REAL server-side pipeline)
Method: logged in and POSTed `/api/backup/tier2` with a real session + CSRF token — the exact endpoint
the UI button invokes. No engine internals were called directly.
**Box state:** sharing ON, 2 shares (`dokumentumok`, `filmek`, both „Felhőmentés" ON) on
`/mnt/felhom-drives/hdd_1` (`/dev/sdb`); second schedulable drive `scratch1` (`/dev/sdd`) — genuinely
different physical disks.
**Tier-2 leg — PASS.**
```
/opt/docker/stacks/samba/smb.conf -> /etc/samba/smb.conf rw=false
/mnt/felhom-drives/hdd_1/media/filmek -> …/media/filmek rw=false (read-only share)
/mnt/felhom-drives/hdd_1/shares/dokumentumok -> …/shares/dokumentumok rw=true
samba-passdb volume -> /var/lib/samba rw=true
[shares] payload staged: 2 share definition(s), credential copy=true
[shares] tier-2 copied 2 share(s) from /mnt/felhom-drives/hdd_1
/mnt/felhom-drives/scratch1/backups/secondary/_shares (8.1 KB)
```
Listeners LAN-only (docker 172.x bridges absent — `bind interfaces only` working):
`445` on `127.0.0.1` + `192.168.0.127`, `137/138` NetBIOS, `3702` WSD, `5357`.
```
_shares/.felhom-tier2-layout → "2" (marker present, written last)
_shares/_payload/_shares-manifest.json → -rw------- , both shares, correct paths/flags
_shares/_payload/passdb.tar → -rw------- , 855 040 B (real capture)
_shares/mnt_felhom-drives_hdd_1/dokumentumok/… → r7.txt, r7b-live.txt
_shares/mnt_felhom-drives_hdd_1/filmek/
```
A file created *after* deploy (`r7b-live.txt`) was picked up on the next run and is **md5-identical**
to the source (`d6ce02292924d3dfb48cac0ede437225` both sides). Payload permissions survive the rsync
mirror as `0600`.
### 7. Live validation
**Display mapping — PASS (after the finding-(d) fix).**
`Event pushed: crossdrive_completed (info) — Másodlagos mentés elkészült: Megosztások`
**Method:** the UI is password-gated and no browser leg was used, so validation drove the **exact
endpoints the UI posts to** (curl with a real session cookie + `_csrf` token scraped from the rendered
page) from inside guest 9201. No server logic skipped — only client rendering.
**Samba liveness (Scenario E, ON direction) — PASS.** With sharing on, stopping `felhom-samba`:
```
health: {"issues":["Protected container not running: felhom-samba"],"status":"fail"}
11:29:55 Event pushed: health_critical (error) — Rendszer állapot kritikus (volt: ok) hub_status 200
```
The next tick's `EnsureBaseStack` self-healed the container (`Up 32 seconds`), and an earlier cycle
produced the matching `health_recovered`. A first attempt showed no degradation event; investigating
rather than assuming showed `scheduler.Every` waits a full interval before its first run, so that tick
hit the deliberate `prev == ""` first-observation guard — a test-timing artifact, not a defect. The
clean re-run above is the real proof.
| Leg | Evidence |
|---|---|
| login + `GET /sharing` | 302 login; page renders (title + nav present) |
| `POST /sharing/enable` | 303; `smb.enabled=true, server_name=FELHOM` |
| `POST /sharing/password` | 303; `user_set=true`; log `household SMB password applied for user=felhom` |
| `POST /sharing/shares` (new folder) | 303; `…/hdd_1/shares/dokumentumok` created **owned 1000:1000** |
| `POST /sharing/shares` (existing, read-only) | 303; stored `read_only: true` |
| Guard refusals (`/api/sharing/browse`) | `appdata`, `backups`, `/etc`, drive-root → **all 400, identical** `{"error":"Ez a mappa nem osztható meg."}` |
| Guard accept | `…/hdd_1/media` → 200 with entries |
| Generated `smb.conf` | hardened global verbatim; `[filmek] read only = yes`, `[dokumentumok] read only = no`, force-user block on both |
| Windows 11 (192.168.0.110) | `Test-NetConnection 445`**True** |
| NetBIOS flat name (nmbd) | `nbtstat`: `FELHOM <00>/<03>/<20> Registered`; `ping FELHOM``Reply from 192.168.0.127` |
| SMB round-trip (writable) | write → readback **BYTE-COMPARE: PASS** |
| **Write-refused (read-only)** | write to `\\192.168.0.127\filmek` **rejected**; folder still empty on the box (non-effect proven) |
| `force user` promise | SMB-written `r7.txt` owned **1000:1000** on the box |
| WS-Discovery | ProbeMatch from `192.168.0.127` **PASS** |
| **Explorer (human, Viktor)** | both shares open from the Network view; interactive Explorer **save** landed owned `1000:1000`; write into the read-only share **refused**, folder still empty |
### NOT live-exercised — stated explicitly
#### Two real bugs caught by live validation (fixed + regression-tested)
- **The offsite `_shares` restic leg** and **the „Megosztások" restore round-trip (Scenario D)**. The
demo box has **no offsite target configured** (`offbox_target` empty — no Storage Box credentials,
no escrow), and the run gate refuses offsite work until the repo password is escrowed. Configuring
one is operator/Viktor territory ("first offsite run = Viktor"). Both are unit-covered and
red-proofed (red-proofs 2, 3, 4), but neither has touched a real restic repo.
- **Scenario E's OFF direction** was not live-toggled — disabling sharing on the demo box would down
the stack and churn customer state. Unit-tested in both directions with red-proof 5.
- **Scenario F on a genuinely disconnected drive** (unit-tested; the live shares are on healthy drives).
1. **`/api/sharing/browse` unreachable** — the `/api/` subtree is routed on the main mux in `main.go`,
so the case added to the web `ServeHTTP` switch was shadowed by the apiRouter catch-all and
returned `401 authentication required`. Fixed by registering
`mux.Handle("/api/sharing/", RequireAuth(CsrfProtect(ServeSharingAPI)))`, matching `/api/storage/`.
2. **Share creation silently failed** — the "new folder" flow passed the storage ROOT through
`sharingResolvePath`, which correctly refuses the drive root as a share TARGET. Split out
`sharingResolveStorageRoot` (accepts exactly a registered live root — strictly tighter);
`TestSharingResolveStorageRoot` asserts both halves.
## 8. Handoff
### 8. NOT yet live-validated
- ~~Explorer render~~ — **PASSED 2026-07-18 (Viktor).** Network → FELHOM → both shares open. He went
further than the scripted leg: a real Explorer **save** into `dokumentumok` succeeded and landed on
the box owned **1000:1000** (so `force user` holds for an interactive Explorer write, not just a
scripted one), and a write into the read-only `filmek` was refused by Windows
("Destination Folder Access Denied") with the folder left **empty** on disk. Slice 1 is fully
PROVEN-LIVE; the capability-map row is flipped.
- **Share data in a real backup run** — blocked by the §4 design fork (classified, not executed).
- **Symlink-escape guard on Linux** — unit test SKIPs on Windows (no privilege).
- Drive-disconnect behaviour live (unit-tested only); multi-root share sets; SMB throughput/signing.
### 9. Observations (not acted on)
- `monitor.EffectiveProtected` derives only from `cfg.Stacks.Protected` (golden-generated), so samba is
**not** liveness-monitored — a dead samba container raises no "protected container missing" issue.
Fixing it needs the golden controller.yaml to list samba, or the SMB-enabled flag threaded through 4
`RunHealthCheck` call sites. Deferred deliberately; the shipped direction is the safe one (no false
alarms while the feature is off).
- A classified bind's `RelPath` is relative to its owning storage root; a share set spanning
**multiple** roots has no single resolution root in the current `(Root, RelPath) + hddPath`
vocabulary — part of the §4 fork.
- `sambaWriteAtomic` is a fourth atomic-write helper (with `backup.atomicWrite`,
`bootstrap.writeFileAtomic`, `setup.atomicWriteFile`) — a shared util is worth a cleanup pass.
- A throwaway `r7.txt` from the round-trip test remains in the demo `dokumentumok` share (harmless;
left so the Explorer leg has visible content).
- **Viktor: raise the managed-update floor to v0.145.0** (supersedes the earlier 0.144 note) so the
N100 rehearsal's day-0 box converges onto the honest version.
- **Viktor: the offsite leg + restore round-trip need a live leg** on a box with an escrowed offsite
target. That is the one part of R-7b whose real-world behaviour rests on tests alone.
- R-7 slice-2 remainder is unchanged (avahi/`.local`, app-folder presets, per-share users, recycle bin).