docs(R-7b): share backup EXECUTION shipped (controller v0.145.0, Model B')
- capability map: SMB row KNOWN GAP cleared -> share data rides both tiers; the offsite leg + restore round-trip flagged as not-yet-live-exercised - ROADMAP R-7b: idea -> SHIPPED, with the Model B' rationale and the live evidence - controller/sharing.md: the KNOWN GAP block replaced by the execution contract; operator note corrected — samba IS liveness-monitored since v0.145.0
This commit is contained in:
@@ -85,7 +85,7 @@
|
||||
| LAN access when internet is down (lan_resolver) | agent | **IMPLEMENTED** | — | Never drilled as a customer experience ("net down — can I reach my photos?") → R-19 |
|
||||
| Phone photo backup | immich (classified) | **PROVEN-LIVE** | 6D end-to-end restore proof | |
|
||||
| Documents/OCR | paperless-ngx (classified) | **PROVEN-LIVE** | `CAMPAIGN-6C` 4P-1 (deploy paperless-ngx, ingest 3 docs via consume flow, OCR + PDF/A ~90s) + 4P-2/3/5 | Consume-folder ingestion awkward without SMB → R-7 |
|
||||
| Files from Windows Explorer / Mac Finder (SMB **server**) | controller v0.144.0 + `felhom-samba:1.0.0` | **PROVEN-LIVE** | `felhom-controller` REPORT.md (v0.144.0) + `controller/sharing.md`; transport verdict `audits/SPIKE-lan-discovery-2026-07-18.md` | „Megosztás" page: enable + one household password + shares (new folder or picked existing, per-share read-only). Fourth protected infra stack (host-net, smbd+**nmbd**+wsdd). Live on demo: 445 reachable, NetBIOS `FELHOM` resolves, write/read byte-compare PASS, write to a read-only share REFUSED, SMB writes land as uid 1000. **Explorer leg PASSED 2026-07-18 (Viktor):** Network → FELHOM → both shares open; a real Explorer *save* into `dokumentumok` landed owned uid 1000, and a write into the read-only `filmek` was refused by Windows with the folder left untouched. KNOWN GAP: share data is backup-**classified** (mandatory/optional per share) but not yet in a live tier-2/offsite RUN — reported design fork, see REPORT §4 |
|
||||
| Files from Windows Explorer / Mac Finder (SMB **server**) | controller v0.145.0 + `felhom-samba:1.0.0` | **PROVEN-LIVE** | `felhom-controller` REPORT.md (v0.144.0) + `controller/sharing.md`; transport verdict `audits/SPIKE-lan-discovery-2026-07-18.md` | „Megosztás" page: enable + one household password + shares (new folder or picked existing, per-share read-only). Fourth protected infra stack (host-net, smbd+**nmbd**+wsdd). Live on demo: 445 reachable, NetBIOS `FELHOM` resolves, write/read byte-compare PASS, write to a read-only share REFUSED, SMB writes land as uid 1000. **Explorer leg PASSED 2026-07-18 (Viktor):** Network → FELHOM → both shares open; a real Explorer *save* into `dokumentumok` landed owned uid 1000, and a write into the read-only `filmek` was refused by Windows with the folder left untouched. Share data now RIDES BOTH BACKUP TIERS (R-7b, controller v0.145.0, Model B′ sibling shares source): tier-2 cross-drive legs + an offsite `_shares` restic snapshot carrying the share definitions and the credential copy, with a „Megosztások" restore. Tier-2 leg + samba liveness alert PROVEN-LIVE on demo; the offsite leg and the restore round-trip are unit-covered but NOT yet live-exercised (the demo box has no offsite target) |
|
||||
| Media to TV via DLNA | — | **MISSING** | — | Jellyfin app exists; DLNA/SSDP unvalidated → R-6, R-8 |
|
||||
| File access via browser | FileBrowser (infra app, auto-mount sync) | **IMPLEMENTED** | FileBrowser runs healthy + userdata-bound (`storage-lifecycle-acceptance-2026-06-15`, `CAMPAIGN-3`) | Actual browse/download **through FileBrowser** is exercised in no doc. (Cited `CAMPAIGN-2` T-PAGE-ALL renders only the controller dashboard pages, not FileBrowser.) Demoted |
|
||||
| Forgot dashboard password → instant reset code | controller v0.123, hub | **PROVEN-LIVE** | `DRILL-day0-take2-2026-07-12` F-15 (live re-run of the exact failure path: hash applied 1s after request, code accepted first try) | |
|
||||
|
||||
@@ -32,7 +32,7 @@
|
||||
| R-5 | Hub: offsite storage visibility — RESTIC box aggregate (v0.64.0) + PBS DR datastore (v0.65.0), each with fill + operator alert; Offsite tabs + dual dashboard gauges | M | **SHIPPED (hub v0.64.0 + v0.65.0 + tenantsync v1.2.0, 2026-07-17)** | Restic: `monitor.OffsiteBoxChecker` (throttled Hetzner GET; fill 80/90% + oversub 2.0×; scope `"pool-box"`). PBS DR (v0.65.0, Option A): read-only `usage` op on the ep0 `felhom-tenantsync` forced command (`df`, no mutation) → `monitor.PBSDRBoxChecker` (fill 80/90%, states ok/unavailable/degraded, scope `"pbsdr-box"`); `/offsite` Restic/PBS-DR tabs + two gauges; hub deploy ⟂ ep0 update (graceful "n/a"). **Both data pipelines PROVEN-LIVE** (restic `0.2% full`; PBS `19.1% full, 7.1 GB of 37.2 GB` after ep0 → v1.2.0). Thresholds pending Viktor (named keys; separate restic/PBS pairs). OPEN: UI render unit-only (password-gated), alert emails unit + red-proof only (both nominal). See hub v0.64.0 + v0.65.0 REPORTs |
|
||||
| R-6 | **Spike: LAN service discovery from the guest** — SSDP multicast (UDP 1900, DLNA), WSD (Windows discovery), mDNS; host-network vs macvlan; is the customer LXC LAN-bridged in appliance deployments? | M | **spiked (2026-07-18)** | **VERDICT: appliance guest IS LAN-bridged (own DHCP lease on the household /24); multicast discovery works ONLY in the guest netns — guest-direct or Docker `--network host` (SSDP/mDNS/WSD all PASS both ways); the default docker bridge is categorically DEAF to LAN multicast (WSD/mDNS RX FAIL, unicast-publish PASS). Real samba+wsdd on host-net → Windows 11 ProbeMatch + FELHOM-SPIKE renders in Explorer + 445 + authenticated SMB round-trip all PASS; real SSDP `MediaServer:1` advert reaches both LAN clients. → R-7 SMB stack MUST be host-network LAN-bound; R-8 Jellyfin-DLNA plausible if host-network. Caveat: `vmbr0 multicast_snooping=1` worked only because the household router is a live querier — customer LANs w/ snooping+no-querier, and Peti's BYO bridge, are UNTESTED gaps.** **S4b (human leg, the sharpest finding): wsdd makes the box VISIBLE but the Explorer double-click FAILS `0x80070035` — WSD gives no name resolution; the flat `\\FELHOM-SPIKE` resolved by no path. Adding `nmbd` (NetBIOS) fixed it live (flat name resolves + mounts). → R-7 needs smbd+wsdd+nmbd (+avahi/.local for modern clients), not wsdd alone.** Doc: `audits/SPIKE-lan-discovery-2026-07-18.md`. |
|
||||
| R-7 | SMB **server** share — LAN file sharing as an embedded controller feature („Megosztás") | L | **SHIPPED slice 1 (controller v0.144.0 + `felhom-samba:1.0.0`, 2026-07-18)** | Flips map row *Files from Windows Explorer / Mac Finder (SMB server)* MISSING → **IMPLEMENTED**. Delivered as the FOURTH protected infra stack (host-net, smbd+**nmbd**+wsdd, `bind interfaces only = lo eth0`), NOT a catalog app. „Megosztás" → „Hálózati megosztás": enable + ONE household password (STDIN→smbpasswd, never persisted — only `user_set`), shares table, create flow (new folder under `<storage>/shares/` or an existing folder via a guarded picker). Picker guard = absolute → EvalSymlinks → containment in a registered LIVE root → deny-listed system subtree (derived from `ProtectedHDDPaths`, provably a subset) → is-a-dir, with UNIFORM refusals (no filesystem oracle). Per-share read-only enforced in smb.conf AND as a `:ro` bind. `force user = felhom` → SMB writes land as uid 1000. Live-validated on demo through the real endpoints + a Win11 workstation (445 True, NetBIOS `FELHOM` resolves, write/read byte-compare PASS, **write to a read-only share REFUSED**, WSD ProbeMatch). **Explorer leg PASSED 2026-07-18 (Viktor):** both shares open from the Network view; an Explorer save into the writable share landed as uid 1000 and a write into the read-only share was refused, folder untouched. Slice 1 is fully PROVEN-LIVE. Docs: `controller/sharing.md`; ship report `felhom-controller/REPORT.md`. **Slice 2 remainder:** avahi/`.local` for modern+Apple clients, curated app-folder presets (one-click paperless `consume/`), per-share users + guest access, recycle bin. **Backup gap split out → R-7b.** |
|
||||
| R-7b | **Share data is backup-CLASSIFIED but not backed up** (design fork reported by R-7 slice 1, not improvised). `ClassifiedBinds("samba")` correctly returns mandatory/optional per share, but no live run consumes it: `backup.RunTier2` short-circuits on `os.Stat(unitDir)` BEFORE calling `GetStackClassifiedBinds`, and the offsite runner enumerates `settings.GetOffboxApps()`. Both engines are **recovery-unit shaped**, which a share-only infra stack has not. Decide the model: give samba a synthetic recovery unit, teach the tier engines a unit-less class-driven source, or keep shares tier-2-only. | M | **idea (needs Viktor ruling)** | Origin: R-7 slice 1 (`felhom-controller` REPORT §4). **Customer-facing consequence: until this lands, files dropped on a share are NOT in any backup — do not promise otherwise.** [R4] intended new shares to default to the full product backup promise (mandatory = offsite + tier-2); the classification honours that, the execution does not yet. Also blocks a clean "every SMB-writable path has a backup class" claim in the R-7 docs |
|
||||
| R-7b | **Share backup EXECUTION** — put share data into the live tier-2 + offsite runs (the design fork reported by R-7 slice 1) | M | **SHIPPED (controller v0.145.0, 2026-07-18)** | **Viktor's ruling: Model B′ — a SIBLING shares source.** New, additive job/leg code reusing the proven primitives (tier-2 mirror seam, restic wrappers, soft-quota/enlargement gate, status recorders) while leaving **every per-app engine path byte-identical** — NOT a synthetic recovery unit (breaks on multi-drive shares, wraps 1 KB of JSON in dump machinery) and NOT engine-loop surgery. The B′ invariant is enforced by test in both tiers, red-proofed. Tier 2 → `RunSharesTier2` (legs grouped by SOURCE drive → `backups/secondary/_shares/<driveKey>/<share>`, payload at `_payload/`, layout marker LAST). Tier 3 → `runOffboxSharesLeg`: ONE extra `restic backup --tag felhom-offbox --tag _shares` placed after the app loop and BEFORE retention, so `forget --group-by host,tags` covers the new group with no flag change; a quota-blocked push degrades to the **manifest only, never to nothing**. Restore → „Megosztások" on `/backups/restore`: scratch, then a missing-only merge whose every destination is PREFIX-ASSERTED against live storage roots, definitions merged existing-wins, then `ReconcileSamba`, then the credential. The **payload** (`_shares-manifest.json` + a best-effort secret-bearing `passdb.tar`) is what makes DR return files + configuration + password rather than loose bytes. **Fold-in: samba joins the liveness set** — `EffectiveProtected` adds the CONTAINER `felhom-samba` exactly while sharing is on. **Live-validated on demo:** real `/api/backup/tier2` trigger → `_shares` tree + marker + payload on the cross-drive target, mirrored file md5-identical, payload 0600 preserved; samba stopped → `health_critical` pushed and hub-accepted (200) → self-healed. **NOT live-exercised (no offsite target on the demo box): the restic `_shares` leg and the restore round-trip** — unit-covered + red-proofed; needs a box with an escrowed offsite target (Viktor). Findings: the reserved-name assumption was FALSE (`nbNameRe` accepted „_shares" as a share name — now refused); the alert/e-mail pipeline needed NO change and adds no new event type. Docs: `controller/sharing.md`; ship report `felhom-controller/REPORT.md`. |
|
||||
| R-8 | DLNA (**gate input now exists — R-6 spiked 2026-07-18: SSDP reaches LAN clients from host-net**): validate Jellyfin's built-in DLNA server first; only add minidlna to the catalog if Jellyfin-DLNA fails | S | idea (unblocked) | Don't add catalog weight before proving the cheap path. **R-6 confirmed the cheap path is physically viable — Jellyfin DLNA must run host-network (same multicast constraint as R-7)** |
|
||||
| R-9 | Uninstaller trio (from 07-15 Peti session): cluster-aware `felhom_guests` guard (node-local `pct list` deletes cluster-wide pveum objects); saferemove detection + time estimate + opt-in `--quick-remove` (never mutate `storage.cfg`); smarter `restore_storage` default for BYO clusters (shared storage, not local-lvm) | M | idea | Second item's rejected alternative (temp-disable-and-restore) stays rejected — crash window silently downgrades cluster wipe policy |
|
||||
| R-10 | T-6E-1: DB-dump dir-fsync asymmetry (LOW, confirmed in 6E) | XS | idea | One-line hardening; batch with the next controller task |
|
||||
|
||||
@@ -142,21 +142,56 @@ The new-folder flow uses a separate, strictly tighter check (`sharingResolveStor
|
||||
|
||||
`smb.conf` and the passdb are config, not customer data, and are never classified.
|
||||
|
||||
> **KNOWN GAP — share data is classified but not yet backed up.** Making this seam correct does not by
|
||||
> itself put share data into a live run: `backup.RunTier2` short-circuits on a missing *recovery unit*
|
||||
> before it ever calls `GetStackClassifiedBinds`, and the offsite runner enumerates
|
||||
> `settings.GetOffboxApps()`. Both engines are recovery-unit shaped, which a share-only infra stack
|
||||
> has not. Teaching them about one is a structural change, so it was reported as a design fork rather
|
||||
> than improvised inside the engines. **Until it is resolved, do not tell a customer that files
|
||||
> dropped on a share are backed up.** See `felhom-controller/REPORT.md` §4.
|
||||
## Backup EXECUTION — the sibling shares source (R-7b, controller v0.145.0)
|
||||
|
||||
The classification above is now *executed*. It did not get there through `ClassifiedBinds` — the
|
||||
engines stayed recovery-unit shaped and Model B′ deliberately left every per-app path byte-identical.
|
||||
Instead `internal/backup` runs a **sibling shares source** off the same registry:
|
||||
|
||||
| Tier | Entry point | What it writes |
|
||||
|---|---|---|
|
||||
| 2 (cross-drive) | `RunSharesTier2`, after the per-stack loop in the same run | `backups/secondary/_shares/<sourceDriveKey>/<share>` per source drive, `_payload/`, `.felhom-tier2-layout` written **LAST** |
|
||||
| 3 (offsite) | `runOffboxSharesLeg`, after the per-app loop and **before** retention | ONE `restic backup --tag felhom-offbox --tag _shares` = payload dir + every MANDATORY share |
|
||||
| restore | „Megosztások" on `/backups/restore` | scratch → missing-only merge, each destination prefix-asserted against LIVE storage roots |
|
||||
|
||||
**The payload is the point.** `_shares-manifest.json` carries the share *definitions* (sorted →
|
||||
byte-deterministic, so an unchanged registry gives the mirror nothing to rewrite) and `passdb.tar`
|
||||
carries the household credential, best-effort. Without it a restore hands the customer their files
|
||||
back and an empty „Megosztás" page. The credential copy is **secret-bearing**: 0600, encrypted inside
|
||||
restic, never logged at INFO and never in a report or a committed file.
|
||||
|
||||
**Degradation contract.** A quota-blocked offsite push falls back to the **manifest only, never to
|
||||
nothing** — definitions protection must not regress because the files stopped fitting.
|
||||
|
||||
**Restore semantics.** Files merge missing-only (a live file is never overwritten). Definitions merge
|
||||
**existing-wins**: a restore must never silently flip a live share's read-only or „Felhőmentés"
|
||||
setting, and anything skipped is named in the flash. Then `ReconcileSamba` re-renders `smb.conf`, then
|
||||
the credential goes back into the named volume.
|
||||
|
||||
**`_shares` is a reserved key** (restic tag, dest root, status record). `ValidateSMBShareName` refuses
|
||||
a leading underscore — note this was a REAL gap, not a formality: `nbNameRe` begins with
|
||||
`[A-Za-z0-9_]`, so „_shares" was previously an accepted share name. The key never reaches a customer
|
||||
surface; `backup.DisplayStackName` maps it to „Megosztások" at the notification and prose boundaries,
|
||||
while the persisted set, the tag and the paths keep the raw key.
|
||||
|
||||
> **Live-validation status (2026-07-18).** The tier-2 leg is PROVEN-LIVE on the demo box through the
|
||||
> real `/api/backup/tier2` endpoint (tree + marker + payload on the cross-drive target, mirrored file
|
||||
> md5-identical, payload 0600 preserved). **The offsite `_shares` leg and the restore round-trip are
|
||||
> unit-covered and red-proofed but NOT yet live-exercised** — the demo box has no offsite target
|
||||
> configured. They need a box with an escrowed offsite repo.
|
||||
|
||||
## Operator notes
|
||||
|
||||
- samba is protected in **code** (`config.alwaysProtectedStacks`), because `cfg.Stacks.Protected` comes
|
||||
from the golden-generated `controller.yaml` and predates the feature. Consequence:
|
||||
`monitor.EffectiveProtected` does **not** monitor samba liveness — a dead samba container raises no
|
||||
"protected container missing" issue. Deliberate for slice 1 (the safe direction: no false alarms
|
||||
while the feature is off).
|
||||
from the golden-generated `controller.yaml` and predates the feature. **Since v0.145.0 (R-7b) it is
|
||||
also monitored for liveness:** `monitor.EffectiveProtected` takes the SMB settings and adds the
|
||||
container `infra.SambaContainerName` exactly while sharing is ON, so a dead sharing service raises
|
||||
the standard protected-container issue → `health_critical` → alert → Hungarian degradation e-mail,
|
||||
while a box that never enabled sharing stays quiet (both directions unit-tested and red-proofed).
|
||||
Note the CONTAINER name (`felhom-samba`) is deliberately not the stack name (`samba`) — the health
|
||||
check docker-inspects container names. PROVEN-LIVE 2026-07-18: stopping the container produced
|
||||
`Protected container not running: felhom-samba`, status `fail`, and a hub-accepted `health_critical`
|
||||
event; the next tick's `EnsureBaseStack` self-healed it.
|
||||
- Discovery depends on the household LAN. The R-6 spike's §7d caveat applies: a customer network with
|
||||
IGMP snooping enabled and **no querier** could prune the discovery multicast, and a BYO topology may
|
||||
put the guest on a bridge with no L2 path to the household LAN. The page carries an honest hint that
|
||||
|
||||
Reference in New Issue
Block a user