diff --git a/CHANGELOG.md b/CHANGELOG.md index b2729e9..832f59c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,33 @@ ## Changelog +### v0.119.0 — storage-health coherence (F8) + mapped_uid validation (F4) (2026-07-12) — MinAgent: 0.81.0 + +Fixes CAMPAIGN-3 (`felhom.eu/documentation/audits/CAMPAIGN-3-2026-07-11.md`) storage-UI findings. +MinAgent UNCHANGED (0.81.0) — controller-only; the §3 design fork took the recommended option **B** +(reuse the shipped v0.117.0 classifier), so no agent change. + +- **F8 (MED) — one classification, two surfaces.** The share row's health used to come only from the + agent's SERVER-LEVEL TCP dial (`server:2049/445`), which stays green when a *single* export is + `exportfs -u`'d — so the row showed benign "Készenlét" while the stacks/dashboard already showed the + stub reality. `networkStorageItems` now FUSES the agent view with the consuming-namespace + classification (`fuseNetHealth` → the same `system.ClassifyPathFS` the stacks stub badge reads): a new + `stub` health state wins over a benign idle/ok when the namespace sees local disk at `Where`; a + whole-server `unreachable` still wins over stub; autofs-healthy / network / inconclusive `unknown` + leave the agent health untouched (never manufacture a fault, never force-mount an idle trigger). The + row badge for `stub` = "Hibás — az alkalmazások nem a NAS-t látják". The row and the stacks/dashboard + badge now derive from ONE classification and can never contradict. +- **F4 (LOW) — mapped_uid/gid validated at the door.** `handleNetStorageAdd` range-checks the container + uid/gid (1..65533) after the `<=0` default, BEFORE the job starts. Out of range → an immediate, + friendly Hungarian 400 ("Az alkalmazás felhasználói azonosítója (uid) érvénytelen…"), nothing + installed — the campaign's `mapped_uid:101000` (a host-side mapped value) previously slipped past the + controller and failed only at the agent with a raw `agent_error`. +- **Live-validated (demo 9201):** F8 — `exportfs -u` while idle + drop-mount → the share row flipped to + `stub`/"Hibás — az alkalmazások nem a NAS-t látják" AND the stacks stub badge showed (4), the two + surfaces AGREE; re-export → row cleared to `ok`/"Elérhető" (healthy idle NOT downgraded). F4 — + `mapped_uid:101000` → 400 + friendly message, registry unchanged; `mapped_uid:1000` passed the range + check. Tests incl. the F8 fusion companion (revert → row idle → fail), the autofs-not-stub guard, and + the F4 boundary (65533 pass / 65534 fail), all green. + ### v0.118.0 — backup integrity: atomic volume dumps (F7) + no single-copy (F6) + stale-primary sweep (F5) (2026-07-12) — MinAgent: 0.81.0 Fixes CAMPAIGN-3 (`felhom.eu/documentation/audits/CAMPAIGN-3-2026-07-11.md`) backup findings. MinAgent diff --git a/CONTEXT.md b/CONTEXT.md index 6eaf855..c83f6a3 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -7,7 +7,20 @@ > > Ask Claude Code: "Please update CONTEXT.md with what we did today" -Last updated: 2026-07-12 (v0.118.0 — backup integrity F7/F6/F5; MinAgent 0.81.0 unchanged) +Last updated: 2026-07-12 (v0.119.0 — storage-health coherence F8 + mapped_uid F4; MinAgent 0.81.0 unchanged) + +> **2026-07-12 — v0.119.0: STORAGE-HEALTH COHERENCE (LIVE on 9201). MinAgent 0.81.0 unchanged.** Fixes +> CAMPAIGN-3 F8+F4. **F8 (MED):** the share row's health came only from the agent's server-level TCP +> dial (blind to a single unexported share) → it showed benign "Készenlét" while the stacks cards +> showed the stub — a contradictory UI. `networkStorageItems`→`fuseNetHealth` now reuses the SAME +> `system.ClassifyPathFS` the stacks stub badge reads (§3 fork = option B, controller-only): a new +> `stub` health (badge "Hibás — az alkalmazások nem a NAS-t látják") overrides idle/ok when the +> namespace sees local disk; `unreachable` still wins over stub; autofs/network/unknown leave agent +> health intact (never force-mount). Row + stacks badge now share ONE classifier → can't contradict. +> **F4 (LOW):** `handleNetStorageAdd` range-checks container uid/gid 1..65533 (`validMappedID`) → +> friendly 400, nothing installed (was raw agent_error on 101000). LIVE: F8 row=stub matching stacks +> badge through an exportfs cut, cleared to ok on re-export; F4 uid 101000→400, uid 1000 passes. +> Seam: `s.classifyFSPath`. Task D (fix-3 alerting + ring revision) still queued. > **2026-07-12 — v0.118.0: BACKUP INTEGRITY (LIVE on 9201). MinAgent 0.81.0 unchanged.** Fixes > CAMPAIGN-3 backup findings (`documentation/audits/CAMPAIGN-3-2026-07-11.md`). **F7 (HIGH) atomic diff --git a/REPORT.md b/REPORT.md index e11a833..18a4012 100644 --- a/REPORT.md +++ b/REPORT.md @@ -1,60 +1,59 @@ -# REPORT — v0.118.0: backup integrity (F7 atomic dumps · F6 no-single-copy · F5 stale sweep) +# REPORT — v0.119.0: storage-health coherence (F8) + mapped_uid validation (F4) -**Date:** 2026-07-12 · **Version:** controller v0.118.0 (from v0.117.0) · **MinAgent:** 0.81.0 (UNCHANGED) -· **Deployed:** guest 9201 (`0.118.0` healthy) · **Source:** `felhom.eu/documentation/audits/CAMPAIGN-3-2026-07-11.md`. +**Date:** 2026-07-12 · **Version:** controller v0.119.0 (from v0.118.0) · **MinAgent:** 0.81.0 (UNCHANGED) +· **Deployed:** guest 9201 (`0.119.0` healthy) · **Source:** `felhom.eu/documentation/audits/CAMPAIGN-3-2026-07-11.md`. + +## §3 design fork — decision + +Took the recommended **option B (controller-only)**: the share row reuses the shipped v0.117.0 +consuming-namespace classifier (`system.ClassifyPathFS`) — the exact ground truth the stacks-page stub +badge already reads. No agent change, no new probing surface, and by construction the row and the +stacks badge can never disagree (single source). Option A (agent-side export-level probe) was not built. ## What shipped -- **F7 (HIGH) — atomic volume dumps.** `backup.DumpAppVolumes` writes the tar to `.tar.tmp`, - fsyncs it, then atomically `os.Rename`s over the restore point ONLY on success (+ best-effort dir - fsync via `atomicPromoteTar`) — mirroring the DB-dump path (`dbdump.go` DumpOne). Any tar - error / timeout / dead-NFS EIO removes only the `.tmp`; the last good `.tar` is byte-untouched. The - `.tar.tmp` is invisible to restore-point/stale scans (ends `.tmp`, not `.tar`); orphan tmps are - swept. New `tarVolume` seam for the red-proof. -- **F6 (LOW) — no single-copy.** `RunAllTier2` no longer skips non-HDD apps → volume-only apps get a - cross-drive tier-2 copy. sys_drive restore-point label is now clear ("Belső SSD (rendszer)"), never - blank. `SingleCopyWarning` (honest HU banner) shows on a single-drive box with no off-drive target. -- **F5 (LOW) — stale primary sweep.** `pruneStalePrimaryDirs` (end of each backup cycle) removes an - orphaned `backups/primary/` dir on an OLD drive after an HDD_PATH move. Guards: deployed + - different-current-drive only; never the current-drive dir or an undeployed app's dir; strictly under - a `backups/primary/` prefix. -- **Part 4 — locality fork: operator chose (A) keep locality, document-only.** No code change; the - NAS-outage behaviour + tier-2 mitigation are documented plainly. +- **F8 (MED) — one classification, two surfaces.** `networkStorageItems` now fuses the agent's health + with the namespace classification via `fuseNetHealth`: a new `stub` state overrides a benign idle/ok + when the consuming namespace sees local disk at `Where`; a whole-server `unreachable` still wins over + stub; autofs-healthy / network / `unknown` leave the agent health intact (no manufactured fault, no + force-mount). Row badge for `stub` = "Hibás — az alkalmazások nem a NAS-t látják". The share row and + the stacks/dashboard badge now derive from ONE classifier. +- **F4 (LOW) — mapped_uid/gid range check at the door.** `handleNetStorageAdd` validates the container + uid/gid (1..65533) after the `<=0` default, before the job — out of range → friendly Hungarian 400, + nothing installed. Catches the campaign's `101000` (a host-side mapped value) that used to leak a raw + `agent_error`. ## Tests + red-proofs (all green) -- F7: happy atomic promote (no `.tmp` left); **truncation red-proof** — pre-seeded good `.tar`, tar - fails mid-write → original byte-unchanged, no 0-byte (revert to in-place write → fails); leftover - `.tar.tmp` swept. -- F6: volume-only app in the tier-2 run set (revert the skip → fails); single-drive honest signal; - sys_drive drive-label non-blank. -- F5: redeployed residue removed; **guard red-proofs** — undeployed app's restore point kept, - current-drive dir kept. -- Existing `TestListRestorePoints_DriveLabel/SSD` updated (blank → clear label, the F6 behaviour). +- F8 fusion table: idle+stub→stub (the contradiction resolved), ok+stub→stub, **idle+autofs→idle** + (the over-eager autofs=stub mutant fails here), unreachable+stub→unreachable (server wins), + idle+unknown→idle (no manufactured fault). End-to-end `networkStorageItems` stub fusion (companion: + drop the fuse call → row shows raw agent health → fail). +- F4: uid 101000 → 400 + friendly message, agent never reached (companion: drop the check → reaches the + agent → fail); 65534 → 400; 1000 / 65533 / 0-defaults pass the range check. -## Live validation (demo 9201, real pipeline, sim-NAS rails — exportfs only) +## Live validation (demo 9201, sim-NAS rails — exportfs only) -- **F7 money-shot:** established good tars (radarr sha `847a97ad…` +4 others), then `exportfs -u` the - NFS target 4 s into a backup during volume dumps → run `success:false`; **all 5 nas-media volume - tars BYTE-IDENTICAL** (clean host-side diff), **no 0-byte file, no leftover `.tar.tmp`**; the next - clean run produced fresh good tars (`success:true`). This is the exact campaign F7 repro, now safe. -- **F6:** after a tier-2 cycle, volume-only `actualbudget` + `seerr` appear under - `felhom-usb/backups/secondary/` (previously none); `actualbudget` snapshot `drive_label` = - "Belső SSD (rendszer)"; single-copy banner correctly absent on the multi-drive demo. -- **F5:** a seeded stale `nas-media/backups/primary/actualbudget` (actualbudget lives on sys_drive) was - swept by the next backup; the real sys_drive dir + radarr's nas-media dir were kept. -- **Restore round-trip:** calibre-web tamper → restore → BYTE-IDENTICAL (53 files), container healthy — - F7's write change did not affect restore. +- **F8 the contradiction, killed:** baseline healthy → row `ok`, no stub badge. `exportfs -u` while idle + + drop the mount → the SHARE ROW showed `health=stub` ("Hibás — az alkalmazások nem a NAS-t látják") + AND the stacks page showed the stub badge (4) — the two surfaces AGREE (previously: row "Készenlét" + + stacks stub = contradiction). `reachable:true` throughout (the server-level dial is still green — the + exact F8 blindness, now correctly overridden). Re-export → row cleared back to `ok`/"Elérhető" + (healthy idle NOT downgraded — the regression). +- **F4:** `mapped_uid:101000` → 400 + the friendly message, registry unchanged (no `c5uid`), no host + unit/dir residue; `mapped_uid:1000` → 200, passed the range check (then failed later at the + unreachable probe as designed, rolled back clean). ## NOT live-validated / standing items -- Single-drive `SingleCopyWarning` live (the demo has enrolled drives) — unit-tested only. -- Task C (F8/F4) and Task D (ring/alerting) remain queued. -- Peti's box (controller 0.113 / agentless-on-proxmox2) reaches 0.118 + agent 0.85 at his next train — - noted for the roadmap; the agentless-on-proxmox2 cluster gap stands. +- `unreachable`-wins live (a genuine server-down IP on a registered share) — unit-tested only; the F8 + live proof used the export-level cut (the actual finding). +- Task D remains queued: fix-3 boot-time app-start-failure alerting + the ring wrap/count revision + (fix-6 6.5-min horizon under load). +- Peti's box (controller 0.113 / agentless-on-proxmox2) reaches 0.119 (+0.85/0.118) at his next train. - No publish/floor movement; agent untouched (MinAgent 0.81.0). ## Box state at wrap -controller 0.118.0 healthy on 9201; all backups consistent (fresh tier-1 + tier-2 incl. volume-only -apps); NAS re-exported; no stale test residue. +controller 0.119.0 healthy on 9201; nas-media healthy + `ok`; registry = nas-media only (no test +residue); all NAS apps healthy; NFS re-exported. diff --git a/controller/README.md b/controller/README.md index ec850b1..ed9cee1 100644 --- a/controller/README.md +++ b/controller/README.md @@ -864,6 +864,19 @@ not just those with HDD data. Non-HDD apps can configure destination, method, an > alkalmazás nem a NAS-t látja", `NetworkStubs`) that WINS over the recoverable `unreachable` badge. > The agent pair (v0.84.0 `ReassertNetworkMounts`) re-arms triggers on guest start; this layer is the > detection net beneath it. +> - **One classification, two surfaces (v0.119.0, CAMPAIGN-3 F8):** the **share row** on the +> `/storage/network` page now reads the SAME classifier. Its health used to come only from the agent's +> server-level TCP dial (`server:2049/445`), which stays green when a *single* export is `exportfs -u`'d +> — so the row showed benign "Készenlét" while the stacks cards showed the stub. `fuseNetHealth` +> (`netstorage_handlers.go`) fuses the agent view with `classifyFSPath(Where)`: a new **`stub`** health +> (badge "Hibás — az alkalmazások nem a NAS-t látják") overrides idle/ok when the namespace sees local +> disk; a whole-server `unreachable` still wins over stub; autofs-healthy / network / `unknown` leave +> the agent health intact (never force-mount an idle trigger). The row and the stacks badge derive from +> ONE classifier and can never contradict. +> - **mapped_uid validated at the door (v0.119.0, F8's sibling F4):** `handleNetStorageAdd` range-checks +> the container uid/gid (**1..65533**; the guest maps ``→`+100000`, so 65534=nobody and a +> host-side value like 101000 must not be entered) after the `<=0` default — out of range → a friendly +> Hungarian 400, nothing installed (previously a raw `agent_error` from the agent). > - **Deploy view truth (v0.117.0, RCA fix 4):** a deployed app's read-only storage select now marks > `selected` by the app's STORED `HDD_PATH` (extra disabled ` (nem elérhető)` option when the > stored path left the schedulable list) — `IsDefault` selects only for NEW deploys. Pre-fix the view