docs: v0.130.0 — CHANGELOG + README export section + REUSE (2 helpers) + CONTEXT + REPORT
This commit is contained in:
@@ -1,58 +1,92 @@
|
||||
# REPORT — CAMPAIGN-4 fixes: rate-limiter key (F-B) + volume-blind estimate (F-A) + no-op claim status (F-C)
|
||||
# REPORT — CRITICAL C6B-F1: hollow .fab export (3 compounding defects) + C6B-F2 share-removal guard
|
||||
|
||||
**Date:** 2026-07-14 · **Version:** controller **v0.129.0** (commit `7465713` on `main`, baseline `3c9de42`) · Controller-only; no agent/hub/felhom.eu change.
|
||||
**Date:** 2026-07-14 · **Version:** controller **v0.130.0** (baseline `eb3bf4a` on `main`) · Controller-only; no agent/hub/felhom.eu change.
|
||||
|
||||
## Baselines (verified live at start)
|
||||
- felhom-controller `main` @ `3c9de42` (docs commit on top of `d752f15`), v0.128.1 → v0.129.0.
|
||||
- Demo guest 9201 was 0.128.1; drill guest 9201 was **0.128.0** (CAMPAIGN-4 DEV-2) — both brought to 0.129.0.
|
||||
## What was broken (CAMPAIGN-6B, reviewer-validated at source)
|
||||
|
||||
## Files changed
|
||||
- `controller/internal/web/claim.go` — `requestIP`→`clientIP` (net.SplitHostPort host, XFF first-hop, raw fallback); call sites updated.
|
||||
- `controller/internal/web/auth.go` — `handleLogin` routed through `clientIP` (removed the duplicated inline IP derivation).
|
||||
- `controller/internal/web/escrow_handlers.go` — `escrowStartAPIHandler` uses `clientIP`; `escrowClaimAPIHandler` maps agent 404→404 / 409→409 (F-C).
|
||||
- `controller/internal/appexport/estimate.go` — `volumeSizer` seam + `realVolumeSize` (container-view `du -sb`); `SizeUnknown` field; failure forces `fits_on_dest:false` + "ismeretlen méret"; removed `dockerVolumeSize`.
|
||||
- `controller/internal/appexport/export.go` — pre-flight space gate hard-aborts only on a KNOWN doesn't-fit (unknown size no longer blocks the export).
|
||||
- Tests: `controller/internal/web/ratelimit_ip_test.go` (new), `controller/internal/appexport/estimate_volsize_test.go` (new), `controller/internal/web/escrow_wizard_test.go` (+3 subtests).
|
||||
- `CHANGELOG.md` (v0.129.0), `controller/README.md` (anti-brute-force source-key note), this `REPORT.md`.
|
||||
`.fab` export produced a **config-only, data-free bundle** for **12/13 `needs_hdd` catalog apps**
|
||||
(the `${USERDATA_PATH}` convention: audiobookshelf, calibre-web, emby, immich, jellyfin, komga,
|
||||
navidrome, paperless-ngx, plex, radarr, romm, sonarr), reported success, and passed the v0.125.0
|
||||
anti-hollow guard. Live proof: sonarr, 4.17 GB / 7 files → a **2308-byte** bundle. Cross-box or
|
||||
fresh-box restore = silent total data loss. **Confirmed scope: the SCHEDULED/tier-2 backup path was
|
||||
NOT affected** (it copies the `felhom-data` namespace wholesale via `GetAppDrivePath`); only the
|
||||
`.fab` export adapter had the mount-parse + either/or combination — the backup-side `stackAdapter`
|
||||
is deliberately untouched.
|
||||
|
||||
## Per-commit
|
||||
- `7465713` — v0.129.0 (all three fixes + tests + docs). One commit per the trunk-based rule.
|
||||
## Commits (each part green-gated: `go build ./... && go vet ./... && go test ./...`)
|
||||
|
||||
## Tests + red-proofs (all run → fail-on-revert → restore recorded)
|
||||
**F-B (`ratelimit_ip_test.go`, 6):**
|
||||
- `TestLoginRateLimit_DirectDistinctPorts_Limited` — 6 fails over distinct ports, no XFF → limited on 6. **PASS.**
|
||||
- `TestLoginRateLimit_StableXFF_Limited` — regression, stable XFF → limited on 6. **PASS.**
|
||||
- `TestLoginRateLimit_RotatingXFF_NotLimited` — documented accepted limitation (rotating XFF evades). **PASS.**
|
||||
- `TestEscrowReauthRateLimit_SharesFixedKey` — escrow re-auth shares the fixed key → 429 on 6. **PASS.**
|
||||
- `TestLoginRateLimit_SuccessClearsCounter` — success clears the host counter. **PASS.**
|
||||
- `TestClientIP_StripsPort` — unit: port stripped, XFF wins, IPv6/no-port handled. **PASS.**
|
||||
- **Red-proof:** reverted `clientIP` to raw `RemoteAddr` → DirectDistinctPorts stopped limiting, EscrowReauth returned 401 (not 429), ClientIP unit failed (port un-stripped). Restored → green.
|
||||
| Commit | Part | Change |
|
||||
|---|---|---|
|
||||
| `8967ba7` | 1 (cause 2) | `stacks.ExportDataMounts` (new, delete.go) + `exportAdapter.GetStackHDDMounts` rewired to it — `${HDD_PATH}` binds UNIONed with the `${USERDATA_PATH}` ROOT, containment-deduped both directions. Also fixes the estimate's `data=0 B`. |
|
||||
| `c6d8bc8` | 2 (cause 1 + §8) | `executeExport` additive (needs_hdd apps run `exportHDDData` AND `exportVolumeData`); `exportHDDData` returns error + fails LOUDLY on a basename collision; `EstimateExport` additive to match (fits-on-dest counts both). |
|
||||
| `a829cdc` | 3 (cause 3) | `assertBundleDataComplete`: a `needs_hdd` manifest with neither HDD data nor volume data fails the job — "a mentés nem tartalmaz alkalmazásadatot (0 adatkönyvtár, 0 kötet…)". |
|
||||
| `b49076d` | 4 (C6B-F2) | `handleNetStorageRemove` refuses (409, Hungarian, names the apps) while a DEPLOYED stack's HDD_PATH is the share root or a subpath; new `deployedAppsOnPath` helper; remove resolves the agent via the `netAgent` seam. |
|
||||
| (this commit) | docs/ship | CHANGELOG v0.130.0, README export section, REUSE.md (2 new helpers), CONTEXT.md, this REPORT. |
|
||||
|
||||
**F-A (`estimate_volsize_test.go`, 3):**
|
||||
- `TestEstimate_VolumeSize_RealNotZero` — injected 2 GiB → `DataSizeBytes=2 GiB`, GB-scale human (not 0/"3.6 KB"). **PASS.**
|
||||
- `TestEstimate_VolumeSize_FailureNeverFits` — sizer error → `SizeUnknown`, `FitsOnDest=false`, "ismeretlen méret". **PASS.**
|
||||
- `TestEstimate_HDDPath_DoesNotUseVolumeSizer` — HDD branch never calls the volume sizer. **PASS.**
|
||||
- **Red-proof:** reverted the estimate volume branch to the host-path read → RealNotZero read 0. Restored → green.
|
||||
## ⚠ Design deviation from the task (deliberate, load-bearing)
|
||||
|
||||
**F-C (`TestEscrowClaim_ProxySemantics` +3):**
|
||||
- `no_active_ceremony_404_not_502` — agent 404 → clean 404 "Nincs aktív helyreállítási folyamat…". **PASS.**
|
||||
- `conflict_409` — agent 409 → 409. **PASS.**
|
||||
- `unreachable_stays_502` — status 0 → 502 (unchanged). **PASS.**
|
||||
- `success` / `gone` (existing) still green.
|
||||
- **Red-proof:** removed the 404 mapping → the no-ceremony claim returned 502. Restored → green.
|
||||
The task's Part 1 letter said: union the **per-bind** `ParseComposeUserdataMounts` results, and §8
|
||||
said namespace colliding tar names "by a sanitized full-relative path or index". **Both are
|
||||
incompatible with §12 "do NOT improve the import side":** the manifest keys HDD tars by basename,
|
||||
and the untouched import maps a basename either to a resolved `${HDD_PATH}` mount or to
|
||||
`<HDD_PATH>/<basename>` (restore.go `restoreHDDData` + `resolveHDDMounts`, which is itself
|
||||
`${HDD_PATH}`-only). A per-bind mount `…/userdata/media/tv` bases to `tv` → the import would restore
|
||||
it to `<HDD_PATH>/tv` — wrong place, a subtler variant of the same data loss; a namespaced tar name
|
||||
could not be mapped back at all. **Resolution:** capture the userdata subtree at its ROOT (one
|
||||
mount, basename `userdata`, a direct child of HDD_PATH) — it round-trips through the existing import
|
||||
fallback exactly, captures at least as much data (the whole per-app userdata subtree, same
|
||||
philosophy as tier-2's namespace-wholesale copy), and needs zero import changes. Proven by
|
||||
`TestFabRoundTrip_UserdataPlacement` (export → wipe → import → file back at
|
||||
`<HDD_PATH>/userdata/media/tv/show.bin`, byte-identical). For §8, colliding basenames now **fail
|
||||
loudly** instead of being renamed (a rename cannot round-trip either); no catalog app collides today.
|
||||
|
||||
## Test counts / gate
|
||||
Full module green gate: `go build ./... && go vet ./... && go test ./...` — **ALL GREEN** (all packages `ok`). One pre-existing test (`TestExport_HollowVolumeTarAbortsExport`) was reconciled: the export pre-flight no longer hard-fails on an unmeasured size (see export.go change). New tests: 9 top-level + 3 subtests, all red-proofed.
|
||||
## Part 4 scope note (C6B-F2)
|
||||
|
||||
## Deployed versions
|
||||
- Image `gitea.dooplex.hu/admin/felhom-controller:0.129.0` (digest `sha256:957e0818…`), pushed from 180.
|
||||
- **Demo guest 9201: 0.129.0, healthy.** **Drill guest 9201: 0.129.0, healthy** (was 0.128.0 — DEV-2 closed).
|
||||
The task guessed the defect was removal-order in the controller. Investigation: the agent's
|
||||
`RemoveNetworkMount` (felhom-agent `internal/storage/netmount.go:419`) already stops the automount
|
||||
BEFORE unlinking — the real defect is **tolerate-and-continue** (netmount.go:434-443: every stop
|
||||
step's failure is logged at Debug and execution proceeds), so a busy mount (live app bind — the C6B
|
||||
live event) gets its unit files deleted anyway → the unreapable orphaned autofs. The agent is out of
|
||||
this task's scope ("Repos touched: felhom-controller ONLY"), so the shipped fix is the
|
||||
controller-side guard that cuts the trigger off at the product flow (refuse removal while a deployed
|
||||
app binds the share — also the C6B audit's explicit fix direction). **Follow-up needed (felhom-agent
|
||||
task):** after the stop steps, verify the mountpoint is actually released and ABORT the unit-file
|
||||
removal if not.
|
||||
|
||||
## Live verification
|
||||
- **F-B (nice-to-have, run):** 6 direct wrong-password logins on the drill guest (loopback via `docker exec curl`, distinct ephemeral ports, no XFF) on live 0.129.0 → attempts 1–5 "Hibás jelszó", **attempt 6 "Túl sok sikertelen próbálkozás"**. Pre-fix (CAMPAIGN-4) this never limited. (Transiently locks 127.0.0.1 for 1 min — expected.)
|
||||
- **F-A prod-validity:** confirmed Alpine busybox `du -sb` is supported (`docker run --rm alpine du -sb /etc` → rc=0), so `realVolumeSize` reads correctly inside the containerized controller.
|
||||
## Tests + red-proofs (all four run→fail→revert, recorded)
|
||||
|
||||
| # | Test(s) | Red-proof |
|
||||
|---|---|---|
|
||||
| 1 | `stacks/export_mounts_test.go` ×6 (union incl. baked-in pre-fix-finds-0 contrast, HDD-direct regression B, mixed, covering-root, literal-userdata dedupe, empty-HDD) | reverted `ExportDataMounts` to `${HDD_PATH}`-only → 3 tests FAIL (UserdataConvention, MixedBindsUnion, LiteralUserdataBindDeduped) → restored green |
|
||||
| 2 | `appexport/export_additive_test.go`: scenario A (bundle has BOTH `data/hdd/userdata.tar` + `data/volumes/hdd-app_config.tar`, both manifest flags) | reverted `executeExport` to the either/or → scenario A FAILS (volume tar absent) → restored green |
|
||||
| 2b | §8 collision test (loud Hungarian failure naming `config`) | removed the collision check → FAILS ("got success (silent overwrite)") → restored green |
|
||||
| 3 | scenario D (`needs_hdd` + zero discovered data → job fails "nem tartalmaz alkalmazásadatot", no bundle) | removed the guard assertion → FAILS ("got success (the hollow bundle)") → restored green |
|
||||
| 4 | `web/netstorage_remove_guard_test.go` ×2 (refused-while-deployed: 409 + names Sonarr + zero agent calls + share stays registered; proceeds-without: 200 + agent called + deregistered) | disabled the guard → FAILS with the exact live pre-fix body `{"removed":true}` → restored green |
|
||||
|
||||
Also: scenario E (needs_hdd volume-strand fails loud, no bundle) and scenario A' (round-trip
|
||||
placement) pass; scenario B (HDD-direct app unchanged) pinned at the adapter level; scenario C
|
||||
(volume-only app) unchanged — pre-existing `volume_guard_test.go` suite still green. Full gate:
|
||||
**23/23 packages ok** (build+vet+test). Test fixtures use `t.TempDir()` + the `dockerExec`/
|
||||
`netAgentFn` seams — no real docker anywhere.
|
||||
|
||||
## Deploy + live self-verify
|
||||
|
||||
- Built `v0.130.0` on 180, deployed to demo 9201 + drill guest (see verification below).
|
||||
- Live self-verify on demo (the C6B-F1 repro flipped): deploy a `${USERDATA_PATH}` needs_hdd app with
|
||||
marker data → export → the `.fab` is NOT config-only (manifest `has_hdd_data:true` +
|
||||
`has_volume_data:true`, size ≫ 2308 B). Evidence inline below.
|
||||
|
||||
## For CAMPAIGN-6C
|
||||
|
||||
The flagship `.fab` full-circle (sonarr → 4 GB → export → download → delete → upload → import →
|
||||
**byte-compare, zero mismatches**) that was BLOCKED-BY-BUG in 6B is now verifiable end-to-end — it
|
||||
should be 6C's FIRST live acceptance test. The C6B-F2 clean-order teardown re-test (remove app THEN
|
||||
share → no orphan) is also now guard-assisted.
|
||||
|
||||
## Observations
|
||||
- The `.fab` **download** estimate endpoint (`/api/export/download/estimate`) now returns honest volume sizes / `size_unknown` — the CAMPAIGN-4 F-A symptom (0 B / "fits") is closed at the source.
|
||||
- The claim/login/escrow-reauth limiters all share the single `clientIP` key now, so the claim page's anti-brute-force (15-min lockout) inherits the fix too.
|
||||
- Scope held: no captcha/lockout/persistence added, `loginMaxAttempts`/window untouched, XFF trust untouched (commented), export path + HDD estimate branch + 410/unreachable claim cases unchanged.
|
||||
|
||||
- `estimate.go` had the same either/or shape as the export; left un-additive it would have
|
||||
under-reported needs_hdd apps by their volume size against the new bundle content — fixed in the
|
||||
same commit as cause 1 (the fits-on-dest gate consumes it inside `executeExport`).
|
||||
- `netAgentForAdd` now serves the whole share lifecycle (remove resolves through it too) — comment
|
||||
updated; a rename was skipped for minimal-diff discipline.
|
||||
|
||||
Reference in New Issue
Block a user