57be785bad
The gate landed v0.125.0 (2026-07-13), realVolumeSize landed v0.129.0 (2026-07-14) -> red for 16 releases unnoticed. Records that the flagged mount is a NAMED VOLUME (the safe shape) so the fix is an allowlist entry, not a rewrite of correct code.
267 lines
17 KiB
Markdown
267 lines
17 KiB
Markdown
# REPORT — most recent implementation
|
||
|
||
## felhom-controller v0.145.0 — R-7b: share data enters the live backup runs (Model B′) + samba liveness — 2026-07-18
|
||
|
||
**Class:** Implementation. **Shipped:** controller **v0.145.0**, deployed on demo guest 9201 and
|
||
**fully PROVEN-LIVE** (tier-2 leg, offsite leg, restore round-trip, samba liveness).
|
||
|
||
> **Revised 2026-07-18** after a close-out runbook: the prior revision wrongly claimed the demo box
|
||
> had no offsite target and that the offsite leg + restore round-trip were unexercised. Both are now
|
||
> live-proven with snapshot evidence; the faulty verification method is dissected in §7b.
|
||
|
||
## 1. Baselines (re-confirmed at session start)
|
||
|
||
| Repo | `main` @ start | Version | → Shipped |
|
||
|---|---|---|---|
|
||
| felhom-controller | `3dfc49e` | v0.144.0 | **v0.145.0** |
|
||
| felhom.eu | `b76dad0` | docs tip | docs only |
|
||
|
||
Trunk-based, explicit-path staging, no Co-Authored-By.
|
||
|
||
> The repo root also held a stale, unrelated `TASK.md` ("Telemetry Debug Section"). It was ignored —
|
||
> the R-7b prompt was the spec.
|
||
|
||
## 2. What shipped, and why it matters
|
||
|
||
Before this release the „Megosztás" page rendered „Felhőmentés: bekapcsolva" for a share whose files
|
||
were **in no backup at all**. `backup.RunTier2` short-circuits on `os.Stat(unitDir)` before the
|
||
classification seam, and the offsite runner enumerates `GetOffboxApps()`; a share-only infra stack has
|
||
neither a recovery unit nor an offbox toggle, so it fell through both engines. The toggle is now true.
|
||
|
||
**Model B′ (Viktor's ruling) and its invariant.** Share data enters the runs through a **sibling
|
||
shares source** — additive job/leg code reusing the proven primitives while leaving **every per-app
|
||
engine path byte-identical**. That invariant is the headline claim and is enforced by test in both
|
||
tiers, each with a red-proof.
|
||
|
||
| Part | File | Shape |
|
||
|---|---|---|
|
||
| 1 payload | `internal/backup/shares_payload.go` | deterministic `_shares-manifest.json` + best-effort `passdb.tar` |
|
||
| 2 tier-2 | `internal/backup/tier2_shares.go` | per-source-drive legs → `backups/secondary/_shares/<driveKey>/<share>`, `_payload/`, marker LAST |
|
||
| 3 offsite | `internal/backup/offbox_shares.go` | ONE `restic backup --tag felhom-offbox --tag _shares`, after the app loop, before retention |
|
||
| 4 restore | `internal/backup/shares_restore.go` | scratch → prefix-asserted missing-only merge → definitions (existing-wins) → `ReconcileSamba` → credential |
|
||
| 5 liveness | `internal/monitor/healthcheck.go` | `EffectiveProtected` gains a settings-backed dynamic extra |
|
||
| 6 UI | `sharing.html`, `backups_restore.html`, handlers | per-tier status lines; „Megosztások" restore entry |
|
||
|
||
**Seams extracted (never forked):** `selectTier2TargetFrom` (source drive supplied explicitly;
|
||
`selectTier2Target` is now a thin wrapper — the headroom math is untouched) and `tier2ReconcileRoots`
|
||
(pure extraction; `tier2Reconcile` calls it with `hdd`/`userdata`).
|
||
|
||
**Degradation contract:** a quota-blocked offsite push falls back to the **manifest only, never to
|
||
nothing** — definitions protection must not regress because the files stopped fitting.
|
||
|
||
## 3. Commits
|
||
|
||
| Hash | Subject |
|
||
|---|---|
|
||
| `c81df55` | Parts 1–2 — payload builder + tier-2 shares job |
|
||
| `85b76e0` | Part 3 — offsite shares leg + B′ isolation proof |
|
||
| `900c870` | Parts 4–6 — restore, samba liveness, UI truth-up |
|
||
| `3b70a9e` | docs — CHANGELOG/CONTEXT/REUSE/README; caveats cleared |
|
||
| `d0c1d77` | fix — reserved key leaked into the `crossdrive_completed` hub event (found live) |
|
||
|
||
felhom.eu: capability-map SMB row, `backlog/ROADMAP.md` R-7b, `controller/sharing.md`.
|
||
|
||
## 4. Findings
|
||
|
||
**(a) The reserved-name assumption in the task was FALSE — guard added.** The task asked me to verify
|
||
at source that slice-1 validation excludes a leading underscore. It does **not**: `settings.nbNameRe`
|
||
begins with `[A-Za-z0-9_]`, so „_shares" was an **accepted share name**. `ValidateSMBShareName` now
|
||
refuses the leading-underscore namespace (on ADD only, so existing shares are never retroactively
|
||
invalidated). Stack names come from the git-synced catalog rather than customer input, so a `_shares`
|
||
STACK is not realistically reachable — but `RunAllTier2` and `RunOffboxBackup` skip one loudly as
|
||
defense in depth rather than let it clobber the shares tree.
|
||
|
||
**(b) Part 5 — the alert pipeline needed no further change.** A missing protected container →
|
||
`report.Issues` → `Status="fail"` → `notifier.NotifyHealthChange` → the **existing** `health_critical`
|
||
event. No new event type is introduced, so the `allowedEventTypes` gotcha does not apply. Verified
|
||
live: the hub accepted the event with HTTP 200.
|
||
|
||
**(c) A real bug, surfaced by test.** `shareSourceDrive` returned a slash-normalised path, which made
|
||
the target selector's source-drive equality check miss — a share group could have targeted **its own
|
||
source drive**, i.e. a same-disk copy pretending to be tier 2. POSIX-only in effect (`ToSlash` is
|
||
identity there), but a genuine defect. Fixed.
|
||
|
||
**(d) A real leak, surfaced by LIVE VALIDATION, not by any unit test.** The first demo run pushed a
|
||
hub event reading „Másodlagos mentés elkészült: **_shares**" — the reserved key reached Hungarian
|
||
customer/operator copy through the `tier2Notify` → `crossdrive_completed` path I had not mapped. Fixed
|
||
at the **source** of the notification (so no future notifier wiring can reintroduce it), with
|
||
`DisplayStackName` at the main.go wiring as idempotent defense in depth, plus a regression test and a
|
||
red-proof. Re-validated live: the event now reads „…: **Megosztások**". This is the clearest argument
|
||
for the live leg — six green red-proofs did not catch it.
|
||
|
||
**(e) Two warning-prose sites need no mapping.** `offbox_capture.go` and `tier2_capture.go` embed a
|
||
raw stack name in Hungarian prose, but the shares source is a sibling and never flows through the
|
||
capture-set helpers, so `_shares` cannot reach them. Verified rather than assumed; no edit made.
|
||
|
||
**(f) Pre-existing gate failure, untouched → now tracked as ROADMAP R-29.**
|
||
`scripts/docker_run_volume_path_gate.py` fails on `internal/appexport/estimate.go:179`. It fails
|
||
identically on the unmodified tree (verified by stashing), predates this work, and was out of scope.
|
||
All six other gates pass. Dated afterwards for the roadmap entry: the gate landed 2026-07-13
|
||
(v0.125.0) and `realVolumeSize` landed 2026-07-14 (v0.129.0, the CAMPAIGN-4 F-A fix), so **the gate
|
||
has been red continuously for 16 releases** without anyone noticing — the second instance of the
|
||
"gate silently red" class after the v0.123.0 Windows read-only-fsync note.
|
||
|
||
The finding itself is benign: the flagged call mounts a **named volume**
|
||
(`-v <volumeName>:/vol:ro`), which is daemon-side with no host path — the *safe* shape, and the same
|
||
pattern as three entries already on the gate's allowlist. It is not the path-strand class the gate
|
||
exists to catch; the v0.129.0 author avoided that class deliberately (the function's comment says so)
|
||
and merely never added the allowlist entry. **The correct fix is an allowlist addition with its WHY,
|
||
not a docker-cp rewrite** — R-29 records that explicitly so nobody "fixes" it by rewriting a call that
|
||
is already correct. The systemic half of R-29 is the one that matters: the gates run only when a human
|
||
remembers to, so a red gate can survive 16 releases of REPORTs claiming green.
|
||
|
||
## 5. Tests
|
||
|
||
`go build ./... && go test ./...` — **green** (23 packages). 25 new/extended cases in
|
||
`internal/backup` + 2 in `internal/monitor`.
|
||
|
||
### Red-proofs — all six run, all fired, all reverted
|
||
|
||
| # | Break introduced | Test that failed |
|
||
|---|---|---|
|
||
| 1 | shares leg appends its paths into the **app's** argv | `TestOffboxSharesLegLeavesAppCallsByteIdentical` — "B′ INVARIANT VIOLATED" |
|
||
| 2 | mandatory→offsite mapping inverted | `TestOffboxSharesLegPushesMandatoryShare` + `…ExcludesOptionalShare` |
|
||
| 3 | manifest-only degradation dropped (skip leg when blocked) | `TestOffboxSharesLegQuotaDegradesToManifestOnly` |
|
||
| 4 | prefix-assert removed (`liveShareRootOK` → true) | `TestSharesRestoreRefusesDestinationOutsideLiveRoots` — "PLACE GUARD BREACHED" ×2 |
|
||
| 5 | dynamic samba extra removed | `TestEffectiveProtectedTracksSharingToggle` (enabled case) |
|
||
| 6 | shares `destBase` drops the reserved segment | `TestSharesTier2LeavesPerAppTreeUntouched` |
|
||
|
||
Plus the post-hoc regression: passing the raw key to `tier2Notify` fails
|
||
`TestSharesTier2NotifierNeverLeaksReservedKey`.
|
||
|
||
## 6. Deployment
|
||
|
||
```
|
||
gitea.dooplex.hu/admin/felhom-controller:0.145.0 Up (healthy) guest 9201
|
||
```
|
||
Built on 180 (`/mnt/5_hdd/felhom.eu/build/felhom-controller`, explicit pull first), deployed via the
|
||
bootstrap service. Deployed twice — the second time carrying the finding-(d) fix.
|
||
|
||
## 7. Live validation (demo, through the REAL server-side pipeline)
|
||
|
||
Method: logged in and POSTed `/api/backup/tier2` with a real session + CSRF token — the exact endpoint
|
||
the UI button invokes. No engine internals were called directly.
|
||
|
||
**Box state:** sharing ON, 2 shares (`dokumentumok`, `filmek`, both „Felhőmentés" ON) on
|
||
`/mnt/felhom-drives/hdd_1` (`/dev/sdb`); second schedulable drive `scratch1` (`/dev/sdd`) — genuinely
|
||
different physical disks.
|
||
|
||
**Tier-2 leg — PASS.**
|
||
```
|
||
[shares] payload staged: 2 share definition(s), credential copy=true
|
||
[shares] tier-2 copied 2 share(s) from /mnt/felhom-drives/hdd_1
|
||
→ /mnt/felhom-drives/scratch1/backups/secondary/_shares (8.1 KB)
|
||
```
|
||
```
|
||
_shares/.felhom-tier2-layout → "2" (marker present, written last)
|
||
_shares/_payload/_shares-manifest.json → -rw------- , both shares, correct paths/flags
|
||
_shares/_payload/passdb.tar → -rw------- , 855 040 B (real capture)
|
||
_shares/mnt_felhom-drives_hdd_1/dokumentumok/… → r7.txt, r7b-live.txt
|
||
_shares/mnt_felhom-drives_hdd_1/filmek/
|
||
```
|
||
A file created *after* deploy (`r7b-live.txt`) was picked up on the next run and is **md5-identical**
|
||
to the source (`d6ce02292924d3dfb48cac0ede437225` both sides). Payload permissions survive the rsync
|
||
mirror as `0600`.
|
||
|
||
**Display mapping — PASS (after the finding-(d) fix).**
|
||
`Event pushed: crossdrive_completed (info) — Másodlagos mentés elkészült: Megosztások`
|
||
|
||
**Samba liveness (Scenario E, ON direction) — PASS.** With sharing on, stopping `felhom-samba`:
|
||
```
|
||
health: {"issues":["Protected container not running: felhom-samba"],"status":"fail"}
|
||
11:29:55 Event pushed: health_critical (error) — Rendszer állapot kritikus (volt: ok) hub_status 200
|
||
```
|
||
The next tick's `EnsureBaseStack` self-healed the container (`Up 32 seconds`), and an earlier cycle
|
||
produced the matching `health_recovered`. A first attempt showed no degradation event; investigating
|
||
rather than assuming showed `scheduler.Every` waits a full interval before its first run, so that tick
|
||
hit the deliberate `prev == ""` first-observation guard — a test-timing artifact, not a defect. The
|
||
clean re-run above is the real proof.
|
||
|
||
**Offsite `_shares` leg — PASS (live).** The demo box HAS a working offsite target: Hetzner Storage
|
||
Box `u629488-sub1.your-storagebox.de:23`, repo `/home/felhom-repo`, `escrow_state: escrowed`,
|
||
`quota_gb: 50`. Viktor's manual run at **2026-07-18T12:18:16Z** produced snapshot **`e0b9d723`**
|
||
(tags `felhom-offbox,_shares`) carrying exactly the designed three paths — the payload staging dir
|
||
plus both share folders. A second run I triggered through the „Távoli mentés" button endpoint
|
||
(`POST /backup/offbox/run`) produced **`4e2b15ec`** at 12:25:52. Inside it:
|
||
|
||
```
|
||
-rw-r--r-- 1000 1000 39 …/shares/dokumentumok/r7b-restore-probe.txt ← uid 1000 preserved
|
||
-rw------- 0 0 418 …/data/shares-payload/_shares-manifest.json
|
||
-rw------- 0 0 855040 …/data/shares-payload/passdb.tar ← credential present
|
||
```
|
||
The payload dir rides as `drwx------`; both payload files as `0600`. `shares_last_status: ok`,
|
||
`shares_last_count: 2`.
|
||
|
||
**Restore round-trip (Scenario D) — PASS (live, through the real routes).** Probe created (uid 1000,
|
||
md5 `8824dcb5b33cf3583c391860d427bfd0`) → captured in `4e2b15ec` → then **damaged**: probe deleted
|
||
from the share folder, and the `dokumentumok` DEFINITION deleted via the real „Megosztás" endpoint
|
||
(`POST /sharing/shares/delete`), which correctly removed it from the registry and from `smb.conf`
|
||
while leaving the folder and its other files intact. Then `POST /backup/shares/restore` →
|
||
`POST /backup/shares/place`:
|
||
|
||
```
|
||
[shares] restore placed: 1 file(s), 1 definition(s) re-added, 1 kept, 0 refused, credential=true
|
||
[samba] applying samba stack: shares=2 config_changed=true
|
||
```
|
||
- probe back, **md5 identical**, uid 1000 and original mtime preserved;
|
||
- **only 1 file written** — `r7.txt` and `r7b-live.txt` already existed and were NOT overwritten, so
|
||
the missing-only merge is proven on live data, not just in a fake;
|
||
- `dokumentumok` back with its **original** flags (`read_only=false`, `offsite=true`,
|
||
`created_at 2026-07-18T09:55:58Z` — the original stamp, not a fresh one);
|
||
- `filmek` reported **kept** (existing-wins) and its flags/mtime untouched;
|
||
- `smb.conf` re-rendered: `[global] [filmek] [dokumentumok]`;
|
||
- **0 refused**, and nothing written outside the registered roots (only `backups/` changed mtime —
|
||
the scratch, which was removed after successful placement).
|
||
|
||
**Credential leg — PASS for continuity; positive auth is a HUMAN leg.** After the restore the passdb
|
||
account is present and enabled (`Unix username: felhom`, flags `[U]`) with **`Password last set: Sat,
|
||
18 Jul 2026 09:55:58 UTC`** — the original stamp, unchanged, proving the credential path did not
|
||
regress. Negative control: `smbclient -U felhom%WRONG` → `NT_STATUS_LOGON_FAILURE`, so smbd is
|
||
genuinely authenticating. A positive `net use \\FELHOM\dokumentumok` with the real household password
|
||
remains **Viktor's leg** — that password is never persisted by design, so I cannot hold it.
|
||
|
||
**R-7b is therefore fully PROVEN-LIVE:** tier-2 leg, offsite leg, restore round-trip, and samba
|
||
liveness all exercised on the demo box through real endpoints.
|
||
|
||
### Still NOT live-exercised — stated explicitly
|
||
|
||
- **Scenario E's OFF direction** was not live-toggled — disabling sharing on the demo box would down
|
||
the stack and churn customer state. Unit-tested in both directions with red-proof 5.
|
||
- **Scenario F on a genuinely disconnected drive** (unit-tested; the live shares are on healthy drives).
|
||
- **Scenario C's quota degradation live** — the repo is at 2.6 MB of a 50 GB quota, so the gate cannot
|
||
trip without fabricating state. Unit-covered with red-proof 3.
|
||
|
||
## 7b. Root cause — why the previous REPORT claimed the opposite, and what the correct check is
|
||
|
||
The previous revision of this report asserted "no offsite target configured (`offbox_target` empty —
|
||
no Storage Box credentials, no escrow), and the run gate refuses offsite work until the repo password
|
||
is escrowed." Every clause was false, on a box that had run a successful offsite backup minutes
|
||
earlier. **What I actually checked:** I read
|
||
`/var/lib/docker/volumes/felhom-controller-data/_data/data/settings.json` with a Python one-liner
|
||
doing `o = d.get('offbox_target') or {}` and printing `o.get(k)` for six fields. **What it returned:**
|
||
`None` for all six. **Why that was wrong:** the JSON key is `offbox`, not `offbox_target` — pinned at
|
||
`internal/settings/settings.go:96`, `Offbox *OffboxTarget \`json:"offbox,omitempty"\``. My `.get()`
|
||
silently produced a missing key, the `or {}` fallback turned that into an empty dict, and every field
|
||
read back `None`. I then interpreted "all fields None" as "not configured" — when a dict whose
|
||
*every* field is None is overwhelming evidence that the KEY is wrong, not that the values are empty.
|
||
I also asserted the escrow gate's behaviour without ever reading the gate's input (`escrow_state` was
|
||
`escrowed`). **The correct check — the one Phase A used:** resolve the persisted key from the struct
|
||
tag at source (`grep 'Offbox \*OffboxTarget' internal/settings/settings.go`), print the JSON's
|
||
top-level keys before indexing into it, and corroborate against the running system's own surfaces
|
||
rather than a guessed schema — here `restic snapshots --tag _shares` through the controller's own
|
||
restic environment, the `/backups/remote` page, and this release's own `shares_last_*` fields, any
|
||
one of which would have contradicted the claim instantly. **The rule going forward:** a live-state
|
||
claim in a REPORT gets verified the way campaigns verify one — through the system's own surface, from
|
||
at least two independent angles, with a negative result treated as a hypothesis to disprove rather
|
||
than a finding to publish.
|
||
|
||
## 8. Handoff
|
||
|
||
- **Viktor: raise the managed-update floor to v0.145.0** (supersedes the earlier 0.144 note) so the
|
||
N100 rehearsal's day-0 box converges onto the honest version.
|
||
- **Viktor: the SMB positive-auth leg** — `net use \FELHOM\dokumentumok` with the real household
|
||
password, to close the credential path end-to-end. Continuity is proven (passdb account enabled,
|
||
password-last-set stamp unchanged by the restore, wrong password correctly rejected), but the
|
||
password is never persisted by design so the positive test needs a human.
|
||
- R-7 slice-2 remainder is unchanged (avahi/`.local`, app-folder presets, per-share users, recycle bin).
|