Files
felhom-controller/REPORT.md
T
admin 57be785bad docs(R-7b): finding (f) now points at ROADMAP R-29, with the dating and diagnosis
The gate landed v0.125.0 (2026-07-13), realVolumeSize landed v0.129.0 (2026-07-14)
-> red for 16 releases unnoticed. Records that the flagged mount is a NAMED VOLUME
(the safe shape) so the fix is an allowlist entry, not a rewrite of correct code.
2026-07-18 15:17:12 +02:00

267 lines
17 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# REPORT — most recent implementation
## felhom-controller v0.145.0 — R-7b: share data enters the live backup runs (Model B) + samba liveness — 2026-07-18
**Class:** Implementation. **Shipped:** controller **v0.145.0**, deployed on demo guest 9201 and
**fully PROVEN-LIVE** (tier-2 leg, offsite leg, restore round-trip, samba liveness).
> **Revised 2026-07-18** after a close-out runbook: the prior revision wrongly claimed the demo box
> had no offsite target and that the offsite leg + restore round-trip were unexercised. Both are now
> live-proven with snapshot evidence; the faulty verification method is dissected in §7b.
## 1. Baselines (re-confirmed at session start)
| Repo | `main` @ start | Version | → Shipped |
|---|---|---|---|
| felhom-controller | `3dfc49e` | v0.144.0 | **v0.145.0** |
| felhom.eu | `b76dad0` | docs tip | docs only |
Trunk-based, explicit-path staging, no Co-Authored-By.
> The repo root also held a stale, unrelated `TASK.md` ("Telemetry Debug Section"). It was ignored —
> the R-7b prompt was the spec.
## 2. What shipped, and why it matters
Before this release the „Megosztás" page rendered „Felhőmentés: bekapcsolva" for a share whose files
were **in no backup at all**. `backup.RunTier2` short-circuits on `os.Stat(unitDir)` before the
classification seam, and the offsite runner enumerates `GetOffboxApps()`; a share-only infra stack has
neither a recovery unit nor an offbox toggle, so it fell through both engines. The toggle is now true.
**Model B (Viktor's ruling) and its invariant.** Share data enters the runs through a **sibling
shares source** — additive job/leg code reusing the proven primitives while leaving **every per-app
engine path byte-identical**. That invariant is the headline claim and is enforced by test in both
tiers, each with a red-proof.
| Part | File | Shape |
|---|---|---|
| 1 payload | `internal/backup/shares_payload.go` | deterministic `_shares-manifest.json` + best-effort `passdb.tar` |
| 2 tier-2 | `internal/backup/tier2_shares.go` | per-source-drive legs → `backups/secondary/_shares/<driveKey>/<share>`, `_payload/`, marker LAST |
| 3 offsite | `internal/backup/offbox_shares.go` | ONE `restic backup --tag felhom-offbox --tag _shares`, after the app loop, before retention |
| 4 restore | `internal/backup/shares_restore.go` | scratch → prefix-asserted missing-only merge → definitions (existing-wins) → `ReconcileSamba` → credential |
| 5 liveness | `internal/monitor/healthcheck.go` | `EffectiveProtected` gains a settings-backed dynamic extra |
| 6 UI | `sharing.html`, `backups_restore.html`, handlers | per-tier status lines; „Megosztások" restore entry |
**Seams extracted (never forked):** `selectTier2TargetFrom` (source drive supplied explicitly;
`selectTier2Target` is now a thin wrapper — the headroom math is untouched) and `tier2ReconcileRoots`
(pure extraction; `tier2Reconcile` calls it with `hdd`/`userdata`).
**Degradation contract:** a quota-blocked offsite push falls back to the **manifest only, never to
nothing** — definitions protection must not regress because the files stopped fitting.
## 3. Commits
| Hash | Subject |
|---|---|
| `c81df55` | Parts 12 — payload builder + tier-2 shares job |
| `85b76e0` | Part 3 — offsite shares leg + B isolation proof |
| `900c870` | Parts 46 — restore, samba liveness, UI truth-up |
| `3b70a9e` | docs — CHANGELOG/CONTEXT/REUSE/README; caveats cleared |
| `d0c1d77` | fix — reserved key leaked into the `crossdrive_completed` hub event (found live) |
felhom.eu: capability-map SMB row, `backlog/ROADMAP.md` R-7b, `controller/sharing.md`.
## 4. Findings
**(a) The reserved-name assumption in the task was FALSE — guard added.** The task asked me to verify
at source that slice-1 validation excludes a leading underscore. It does **not**: `settings.nbNameRe`
begins with `[A-Za-z0-9_]`, so „_shares" was an **accepted share name**. `ValidateSMBShareName` now
refuses the leading-underscore namespace (on ADD only, so existing shares are never retroactively
invalidated). Stack names come from the git-synced catalog rather than customer input, so a `_shares`
STACK is not realistically reachable — but `RunAllTier2` and `RunOffboxBackup` skip one loudly as
defense in depth rather than let it clobber the shares tree.
**(b) Part 5 — the alert pipeline needed no further change.** A missing protected container →
`report.Issues``Status="fail"``notifier.NotifyHealthChange` → the **existing** `health_critical`
event. No new event type is introduced, so the `allowedEventTypes` gotcha does not apply. Verified
live: the hub accepted the event with HTTP 200.
**(c) A real bug, surfaced by test.** `shareSourceDrive` returned a slash-normalised path, which made
the target selector's source-drive equality check miss — a share group could have targeted **its own
source drive**, i.e. a same-disk copy pretending to be tier 2. POSIX-only in effect (`ToSlash` is
identity there), but a genuine defect. Fixed.
**(d) A real leak, surfaced by LIVE VALIDATION, not by any unit test.** The first demo run pushed a
hub event reading „Másodlagos mentés elkészült: **_shares**" — the reserved key reached Hungarian
customer/operator copy through the `tier2Notify``crossdrive_completed` path I had not mapped. Fixed
at the **source** of the notification (so no future notifier wiring can reintroduce it), with
`DisplayStackName` at the main.go wiring as idempotent defense in depth, plus a regression test and a
red-proof. Re-validated live: the event now reads „…: **Megosztások**". This is the clearest argument
for the live leg — six green red-proofs did not catch it.
**(e) Two warning-prose sites need no mapping.** `offbox_capture.go` and `tier2_capture.go` embed a
raw stack name in Hungarian prose, but the shares source is a sibling and never flows through the
capture-set helpers, so `_shares` cannot reach them. Verified rather than assumed; no edit made.
**(f) Pre-existing gate failure, untouched → now tracked as ROADMAP R-29.**
`scripts/docker_run_volume_path_gate.py` fails on `internal/appexport/estimate.go:179`. It fails
identically on the unmodified tree (verified by stashing), predates this work, and was out of scope.
All six other gates pass. Dated afterwards for the roadmap entry: the gate landed 2026-07-13
(v0.125.0) and `realVolumeSize` landed 2026-07-14 (v0.129.0, the CAMPAIGN-4 F-A fix), so **the gate
has been red continuously for 16 releases** without anyone noticing — the second instance of the
"gate silently red" class after the v0.123.0 Windows read-only-fsync note.
The finding itself is benign: the flagged call mounts a **named volume**
(`-v <volumeName>:/vol:ro`), which is daemon-side with no host path — the *safe* shape, and the same
pattern as three entries already on the gate's allowlist. It is not the path-strand class the gate
exists to catch; the v0.129.0 author avoided that class deliberately (the function's comment says so)
and merely never added the allowlist entry. **The correct fix is an allowlist addition with its WHY,
not a docker-cp rewrite** — R-29 records that explicitly so nobody "fixes" it by rewriting a call that
is already correct. The systemic half of R-29 is the one that matters: the gates run only when a human
remembers to, so a red gate can survive 16 releases of REPORTs claiming green.
## 5. Tests
`go build ./... && go test ./...`**green** (23 packages). 25 new/extended cases in
`internal/backup` + 2 in `internal/monitor`.
### Red-proofs — all six run, all fired, all reverted
| # | Break introduced | Test that failed |
|---|---|---|
| 1 | shares leg appends its paths into the **app's** argv | `TestOffboxSharesLegLeavesAppCallsByteIdentical` — "B INVARIANT VIOLATED" |
| 2 | mandatory→offsite mapping inverted | `TestOffboxSharesLegPushesMandatoryShare` + `…ExcludesOptionalShare` |
| 3 | manifest-only degradation dropped (skip leg when blocked) | `TestOffboxSharesLegQuotaDegradesToManifestOnly` |
| 4 | prefix-assert removed (`liveShareRootOK` → true) | `TestSharesRestoreRefusesDestinationOutsideLiveRoots` — "PLACE GUARD BREACHED" ×2 |
| 5 | dynamic samba extra removed | `TestEffectiveProtectedTracksSharingToggle` (enabled case) |
| 6 | shares `destBase` drops the reserved segment | `TestSharesTier2LeavesPerAppTreeUntouched` |
Plus the post-hoc regression: passing the raw key to `tier2Notify` fails
`TestSharesTier2NotifierNeverLeaksReservedKey`.
## 6. Deployment
```
gitea.dooplex.hu/admin/felhom-controller:0.145.0 Up (healthy) guest 9201
```
Built on 180 (`/mnt/5_hdd/felhom.eu/build/felhom-controller`, explicit pull first), deployed via the
bootstrap service. Deployed twice — the second time carrying the finding-(d) fix.
## 7. Live validation (demo, through the REAL server-side pipeline)
Method: logged in and POSTed `/api/backup/tier2` with a real session + CSRF token — the exact endpoint
the UI button invokes. No engine internals were called directly.
**Box state:** sharing ON, 2 shares (`dokumentumok`, `filmek`, both „Felhőmentés" ON) on
`/mnt/felhom-drives/hdd_1` (`/dev/sdb`); second schedulable drive `scratch1` (`/dev/sdd`) — genuinely
different physical disks.
**Tier-2 leg — PASS.**
```
[shares] payload staged: 2 share definition(s), credential copy=true
[shares] tier-2 copied 2 share(s) from /mnt/felhom-drives/hdd_1
→ /mnt/felhom-drives/scratch1/backups/secondary/_shares (8.1 KB)
```
```
_shares/.felhom-tier2-layout → "2" (marker present, written last)
_shares/_payload/_shares-manifest.json → -rw------- , both shares, correct paths/flags
_shares/_payload/passdb.tar → -rw------- , 855 040 B (real capture)
_shares/mnt_felhom-drives_hdd_1/dokumentumok/… → r7.txt, r7b-live.txt
_shares/mnt_felhom-drives_hdd_1/filmek/
```
A file created *after* deploy (`r7b-live.txt`) was picked up on the next run and is **md5-identical**
to the source (`d6ce02292924d3dfb48cac0ede437225` both sides). Payload permissions survive the rsync
mirror as `0600`.
**Display mapping — PASS (after the finding-(d) fix).**
`Event pushed: crossdrive_completed (info) — Másodlagos mentés elkészült: Megosztások`
**Samba liveness (Scenario E, ON direction) — PASS.** With sharing on, stopping `felhom-samba`:
```
health: {"issues":["Protected container not running: felhom-samba"],"status":"fail"}
11:29:55 Event pushed: health_critical (error) — Rendszer állapot kritikus (volt: ok) hub_status 200
```
The next tick's `EnsureBaseStack` self-healed the container (`Up 32 seconds`), and an earlier cycle
produced the matching `health_recovered`. A first attempt showed no degradation event; investigating
rather than assuming showed `scheduler.Every` waits a full interval before its first run, so that tick
hit the deliberate `prev == ""` first-observation guard — a test-timing artifact, not a defect. The
clean re-run above is the real proof.
**Offsite `_shares` leg — PASS (live).** The demo box HAS a working offsite target: Hetzner Storage
Box `u629488-sub1.your-storagebox.de:23`, repo `/home/felhom-repo`, `escrow_state: escrowed`,
`quota_gb: 50`. Viktor's manual run at **2026-07-18T12:18:16Z** produced snapshot **`e0b9d723`**
(tags `felhom-offbox,_shares`) carrying exactly the designed three paths — the payload staging dir
plus both share folders. A second run I triggered through the „Távoli mentés" button endpoint
(`POST /backup/offbox/run`) produced **`4e2b15ec`** at 12:25:52. Inside it:
```
-rw-r--r-- 1000 1000 39 …/shares/dokumentumok/r7b-restore-probe.txt ← uid 1000 preserved
-rw------- 0 0 418 …/data/shares-payload/_shares-manifest.json
-rw------- 0 0 855040 …/data/shares-payload/passdb.tar ← credential present
```
The payload dir rides as `drwx------`; both payload files as `0600`. `shares_last_status: ok`,
`shares_last_count: 2`.
**Restore round-trip (Scenario D) — PASS (live, through the real routes).** Probe created (uid 1000,
md5 `8824dcb5b33cf3583c391860d427bfd0`) → captured in `4e2b15ec` → then **damaged**: probe deleted
from the share folder, and the `dokumentumok` DEFINITION deleted via the real „Megosztás" endpoint
(`POST /sharing/shares/delete`), which correctly removed it from the registry and from `smb.conf`
while leaving the folder and its other files intact. Then `POST /backup/shares/restore`
`POST /backup/shares/place`:
```
[shares] restore placed: 1 file(s), 1 definition(s) re-added, 1 kept, 0 refused, credential=true
[samba] applying samba stack: shares=2 config_changed=true
```
- probe back, **md5 identical**, uid 1000 and original mtime preserved;
- **only 1 file written** — `r7.txt` and `r7b-live.txt` already existed and were NOT overwritten, so
the missing-only merge is proven on live data, not just in a fake;
- `dokumentumok` back with its **original** flags (`read_only=false`, `offsite=true`,
`created_at 2026-07-18T09:55:58Z` — the original stamp, not a fresh one);
- `filmek` reported **kept** (existing-wins) and its flags/mtime untouched;
- `smb.conf` re-rendered: `[global] [filmek] [dokumentumok]`;
- **0 refused**, and nothing written outside the registered roots (only `backups/` changed mtime —
the scratch, which was removed after successful placement).
**Credential leg — PASS for continuity; positive auth is a HUMAN leg.** After the restore the passdb
account is present and enabled (`Unix username: felhom`, flags `[U]`) with **`Password last set: Sat,
18 Jul 2026 09:55:58 UTC`** — the original stamp, unchanged, proving the credential path did not
regress. Negative control: `smbclient -U felhom%WRONG``NT_STATUS_LOGON_FAILURE`, so smbd is
genuinely authenticating. A positive `net use \\FELHOM\dokumentumok` with the real household password
remains **Viktor's leg** — that password is never persisted by design, so I cannot hold it.
**R-7b is therefore fully PROVEN-LIVE:** tier-2 leg, offsite leg, restore round-trip, and samba
liveness all exercised on the demo box through real endpoints.
### Still NOT live-exercised — stated explicitly
- **Scenario E's OFF direction** was not live-toggled — disabling sharing on the demo box would down
the stack and churn customer state. Unit-tested in both directions with red-proof 5.
- **Scenario F on a genuinely disconnected drive** (unit-tested; the live shares are on healthy drives).
- **Scenario C's quota degradation live** — the repo is at 2.6 MB of a 50 GB quota, so the gate cannot
trip without fabricating state. Unit-covered with red-proof 3.
## 7b. Root cause — why the previous REPORT claimed the opposite, and what the correct check is
The previous revision of this report asserted "no offsite target configured (`offbox_target` empty —
no Storage Box credentials, no escrow), and the run gate refuses offsite work until the repo password
is escrowed." Every clause was false, on a box that had run a successful offsite backup minutes
earlier. **What I actually checked:** I read
`/var/lib/docker/volumes/felhom-controller-data/_data/data/settings.json` with a Python one-liner
doing `o = d.get('offbox_target') or {}` and printing `o.get(k)` for six fields. **What it returned:**
`None` for all six. **Why that was wrong:** the JSON key is `offbox`, not `offbox_target` — pinned at
`internal/settings/settings.go:96`, `Offbox *OffboxTarget \`json:"offbox,omitempty"\``. My `.get()`
silently produced a missing key, the `or {}` fallback turned that into an empty dict, and every field
read back `None`. I then interpreted "all fields None" as "not configured" — when a dict whose
*every* field is None is overwhelming evidence that the KEY is wrong, not that the values are empty.
I also asserted the escrow gate's behaviour without ever reading the gate's input (`escrow_state` was
`escrowed`). **The correct check — the one Phase A used:** resolve the persisted key from the struct
tag at source (`grep 'Offbox \*OffboxTarget' internal/settings/settings.go`), print the JSON's
top-level keys before indexing into it, and corroborate against the running system's own surfaces
rather than a guessed schema — here `restic snapshots --tag _shares` through the controller's own
restic environment, the `/backups/remote` page, and this release's own `shares_last_*` fields, any
one of which would have contradicted the claim instantly. **The rule going forward:** a live-state
claim in a REPORT gets verified the way campaigns verify one — through the system's own surface, from
at least two independent angles, with a negative result treated as a hypothesis to disprove rather
than a finding to publish.
## 8. Handoff
- **Viktor: raise the managed-update floor to v0.145.0** (supersedes the earlier 0.144 note) so the
N100 rehearsal's day-0 box converges onto the honest version.
- **Viktor: the SMB positive-auth leg** — `net use \FELHOM\dokumentumok` with the real household
password, to close the credential path end-to-end. Continuity is proven (passdb account enabled,
password-last-set stamp unchanged by the restore, wrong password correctly rejected), but the
password is never persisted by design so the positive test needs a human.
- R-7 slice-2 remainder is unchanged (avahi/`.local`, app-folder presets, per-share users, recycle bin).