diff --git a/CONTEXT.md b/CONTEXT.md index 10cb3d0..5b2b96b 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -7,7 +7,19 @@ > > Ask Claude Code: "Please update CONTEXT.md with what we did today" -Last updated: 2026-07-13 night (v0.128.0 — browser .fab upload on /import) +Last updated: 2026-07-13 night (v0.128.1 — USB class-badge suppression + demo PVE dir-storage retirement) + +> **2026-07-13 night — v0.128.1 + demo storage hygiene (ruling F5).** `classTag` suppresses the +> rotational class hint for `type==='usb'` (card already carries the USB tag; hub `ClassHint` +> UNCHANGED; pinned by `TestStorageTemplate_USBClassBadgeSuppressed` + red-proof). Host op on +> demo-felhom: the two pre-intermediary legacy `dir:` storages (`felhom-usb`, `felhom-flash`, +> content=Backup, is_mountpoint) RETIRED via `pvesm remove` after G1/G2/G3 gates all PASSED +> (agent-owned UUID .mount units; both `enrolled` in drive-intents.json; zero /etc/pve refs, empty +> dump/, no customer app on either drive). Post-removal: mounts+binds intact (marker round-trip +> through the guest), `GET /disks` shows both registry-sourced (role+durable-id intact, class +> absent), `pvesm status` clean. **The demo node now matches the fresh-install storage shape** — +> drives are registry+units-sourced only, no legacy PVE dir: storages. v0.128.1 LIVE on demo 9201 +> (drill guest skipped — optional, no behavioral dependency; it runs 0.128.0). > **2026-07-13 night — v0.128.0: CHUNKED BROWSER .FAB UPLOAD on /import (ruling F3: chunked).** > Step-0 probe on the real tunnel PROVED the Cloudflare edge cap (120 MiB POST → edge 413 with diff --git a/REPORT.md b/REPORT.md index 06f8b86..afd72ff 100644 --- a/REPORT.md +++ b/REPORT.md @@ -1,92 +1,95 @@ -# REPORT — v0.128.0: chunked browser .fab upload on /import (tunnel-proof) +# REPORT — demo storage hygiene: USB class-badge suppression (v0.128.1) + legacy PVE dir-storage retirement -**Date:** 2026-07-13 · **Version:** controller v0.128.0 (baseline v0.127.3 `59e4ca7` reconfirmed -against live Gitea; local == origin/main) · Commit `db16371` on `main` · MinAgent unchanged (no -agent coupling — the feature is controller-local fs + HTTP). Implements -TASK-fab-browser-upload (ruling F3 2026-07-13: chunked). +**Date:** 2026-07-13 · **Version:** controller v0.128.1 (commit `d752f15` on `main`) · Host op on +demo-felhom PVE (192.168.0.162), config-level only — no data moved or deleted. +**Sequencing note:** the TASK was numbered v0.127.3 and sequenced BEFORE the .fab upload task, but +v0.127.3 (escrow reveal copy) and v0.128.0 (upload) had already shipped — Part 1 ships as +**v0.128.1**; content unaffected. Baseline at start: `61c1391` (v0.128.0 REPORT), live Gitea == +local. -## Step-0 probe — the Cloudflare edge cap, measured on the REAL tunnel (before any code) +## Part 1 — controller: suppress the class badge for USB drives (ruling F5) -| Probe | Result | -|---|---| -| POST 120 MiB (zeros) → `https://felhom.demo-felhom.eu/step0-cap-probe` | **HTTP 413 "Payload Too Large" from the edge** (`Server: cloudflare`, `CF-RAY a1aa78cd1f04324d-VIE`, cloudflare-branded HTML body). Rejected on the declared Content-Length: only ~1.1 MB left the client, 0.41 s. | -| POST 80 MiB, same target | **HTTP 302 → `/login?next=…`** — the controller's auth redirect, i.e. the origin was reached (edge passage PROVEN). 0.48 s; the origin answered before consuming the body (unauth redirect), so no full-upload throughput datum — the edge decision is header-based, which is the datum that matters. | +`storage.html` `classTag(d)`: `if(d.type==='usb') return '';` ahead of the class branches — one +guard covers both render sites (card badges L~324; metarow L~343, which additionally guards on +`d.class`). Non-USB storages keep the hint. **Hub-report `ClassHint` UNCHANGED** (UI-only +suppression). No existing test pinned the badge → new +`TestStorageTemplate_USBClassBadgeSuppressed` (renders the production template tree, asserts the +guard exists AND precedes the class branches). **Red-proof:** guard line removed → FAIL +`"classTag USB guard missing from the storage page"`; restored → PASS. Green gate: full module +build/vet/test clean; template gates (id/emoji/native-confirm) green. -Gotcha recorded: local DNS (192.168.0.250) overrides `felhom.demo-felhom.eu` to the LAN guest -(192.168.0.139), and curl's default `Expect: 100-continue` lets the server answer before the body -is sent — the honest probe needed `--resolve` onto Cloudflare's public IP (172.67.149.59) plus -`-H "Expect:"`. **Outcome: cap confirmed at ~100 MB → `uploadChunkBytes` = 64 MiB (the spec's -default), server-side per-request cap 96 MiB.** +**Deployed:** image `0.128.1` → demo guest 9201, `Up (healthy)`. Drill guest skipped (task: +optional, no behavioral dependency). -## What shipped +## Part 2 — host op: retire `felhom-usb` + `felhom-flash` PVE dir storages -- `controller/internal/web/handler_export_upload.go` (new) — `POST - /api/export/upload/{init,chunk,finalize,abort}` inside `ServeExportAPI` (inherits the main.go - `RequireAuth(CsrfProtect(...))` mount — nothing added at the mux). Single-flight slot - (netstorage-job shape); init sanitizes to a `[A-Za-z0-9._ -]` base name + mandatory `.fab`, - free-space gate (declared + 1 GiB margin, Hungarian error with both numbers); chunk requires - `offset == received` (else 409 + `received_bytes` echo) and streams via `io.Copy`; finalize = - exact declared size (else 422 + `.part` deleted) → fsync → atomic rename, collision → lowest-free - `"name (N).fab"` computed from the ORIGINAL name (never `(1)(1)`); abort deletes the `.part`. - Startup GC sweeps `*.part-*` on every registered drive (wired in `main.go` after - `SetAppExporter`); 15-min idle timer aborts server-side. No client-side sha256 — deliberate, - rationale in the file header (WebCrypto can't stream-hash multi-GB; `.fab` self-validates at - import). -- `controller/internal/appexport/estimate.go` — `diskFree` exported as `DiskFree` (REUSE.md row - added); web seam `uploadDiskFree` package var. -- `controller/internal/web/templates/app_import.html` — upload zone above the bundle list - (drag-and-drop + "Fájl kiválasztása", accept=".fab"), progress "Feltöltés: {pct}% ({done} / - {total} GB)" + "Megszakítás", spec'd Hungarian error strings; `File.slice` sequential loop with - one retry per chunk re-synced from the 409 echo; on success the page reloads (the EXISTING scan - renders the row — no second lister). Copy-by-hand hint now names BOTH paths. Gates green: - template_id, emoji, native_confirm, mojibake. +### Gate evidence (all PASS) -## Tests (all green; full module `go build && go vet && go test ./...` = 23 packages ok) +**G1 — mount provenance:** both mounts are agent-owned systemd units, independent of the PVE +entries: +- `/mnt/felhom-usb` = `/dev/sdc1` (ext4) via `mnt-felhom\x2dusb.mount` — "`Managed by + felhom-agent`", `What=/dev/disk/by-uuid/da9e7089-cf8e-4617-adcb-a377743fae00`, + `WantedBy=multi-user.target`. +- `/mnt/felhom-flash` = `/dev/sdd1` (ext4) via `mnt-felhom\x2dflash.mount` — same shape, + `What=/dev/disk/by-uuid/81a26531-62d8-408d-812f-a178b1d35310`. +- `/etc/fstab`: no entries for either. The PVE `storage.cfg` blocks carried `is_mountpoint 1` + (checker, not owner) and no `mkdir`. -| Scenario | Test | Result | -|---|---|---| -| A happy path (3 chunks byte-exact, existing scan lists it, no `.part` residue) | `TestFabUpload_HappyPath` | PASS | -| B offset replay → 409 + echo, disk unchanged, re-sync continues | `TestFabUpload_OffsetMismatchRejected` | PASS | -| C traversal neutered inside exports; ext/charset/hidden refused; oversize (faked statfs) → Hungarian msg with 3.0/2.0 GB; 2nd init → 409 | `TestFabUpload_SecurityGates` | PASS | -| C route wrap (unauth POST → 401 on all four paths via the main.go-shaped mount) | `TestFabUpload_MountInheritsAuth` | PASS | -| D finalize size mismatch → 422, `.part` deleted, slot freed | `TestFabUpload_FinalizeSizeMismatch` | PASS | -| E startup GC removes stray `.part`, keeps real bundles; idle-expiry deletes + frees slot | `TestFabUpload_GCAndIdleTimeout` | PASS | -| F collision → `app (1).fab`, re-run → `app (2).fab`, original bytes untouched | `TestFabUpload_CollisionLowestFreeSuffix` | PASS | -| abort deletes `.part` + frees slot | `TestFabUpload_Abort` | PASS | +**G2 — registry membership:** `/var/lib/felhom-agent/drive-intents.json` (read-only) lists both +UUIDs as `"enrolled"` (plus teszt_enroll's). Guest binds live under the drives tree: mp8 +`/mnt/felhom-drives` (no `backup=1`); `/mnt/felhom-drives/felhom-{usb,flash}` are `[/felhom-data]` +subdir mounts of the same devices (Model A). Controller settings register both stable paths +(`Tárhely (felhom-usb)` / `Tárhely (felhom-flash)`). -**Red-proofs (§10, all run + reverted, `git diff` clean):** -- **C** — sanitize bypassed (raw client filename used): `TestFabUpload_SecurityGates` FAILED with - "sanitized upload must land inside the exports dir" (the file materialized at the traversal - target `{drive}/evil.fab`). -- **B** — offset check removed (append regardless): `TestFabUpload_OffsetMismatchRejected` FAILED - with "replay: got 200, want 409". -- **F** — naive overwrite (rename straight to the requested name): - `TestFabUpload_CollisionLowestFreeSuffix` FAILED with `first collision landed as "app.fab", - want "app (1).fab"`. +**G3 — zero references:** `grep -r 'felhom-usb\|felhom-flash' /etc/pve/` → only the two +`storage.cfg` blocks themselves + `/etc/pve/.rrd` metric rings (auto-generated stats, not config). +No jobs.cfg/vzdump/replication; both storages' `dump/` dirs EMPTY. No customer app on either +drive: the only stack referencing them is the protected infra FileBrowser, via the agent-owned +`/mnt/felhom-drives/...` bind chain — unaffected by the PVE entry. (Live re-verified, not from +the screenshot.) -## Deploy + live verification +### storage.cfg blocks removed (before-state, verbatim) -- Image `gitea.dooplex.hu/admin/felhom-controller:0.128.0` - (`sha256:4c943ea1…`) built on 180. -- **Demo guest 9201**: `Up … (healthy)` on 0.128.0; selfupdate log confirms "Current version - 0.128.0 is up to date"; no ERROR/FATAL. **Drill guest** (nested PVE 192.168.0.152, guest 9201): - `Up … (healthy)` on 0.128.0. -- Route live-checked on the deployed box: unauthenticated `POST /api/export/upload/init` → 401 - JSON both via LAN and **through the real Cloudflare edge** (`--resolve` onto 172.67.149.59) — - the new path is deployed and auth+CSRF-wrapped on the customer path. +``` +dir: felhom-usb + path /mnt/felhom-usb + content backup + is_mountpoint 1 -## NOT live-validated (Viktor's ~5-minute leg) +dir: felhom-flash + path /mnt/felhom-flash + content backup + is_mountpoint 1 +``` -The end-to-end multi-GB browser upload through the real tunnel needs a dashboard login (CC has no -customer session). Suggested test: export a real app to `.fab`, download it via the browser, -re-upload it on `/import`, import it — the full circle. Everything beneath the login was -exercised: edge cap probed, endpoints tested A–F, deployed route answers through the tunnel. +### Removal + verification + +Pre-removal marker files created under each drive's felhom-data namespace and confirmed readable +from the guest. Then `pvesm remove felhom-usb && pvesm remove felhom-flash` → OK. Verified in +order: +1. **Host mounts intact**, same sources: `/mnt/felhom-usb` = `/dev/sdc1`, `/mnt/felhom-flash` = + `/dev/sdd1`; drives-tree subdir mounts intact (`/dev/sdc1[/felhom-data]`, + `/dev/sdd1[/felhom-data]`). +2. **Guest binds intact**: both markers still readable from guest 9201 post-removal + (`retire-check-2026-07-13`); markers removed afterwards. +3. **Meghajtók data**: agent `GET /disks` (pinned local-API token, value never echoed) returns + both drives registry-sourced — `type: usb`, `role: user-data`, durable-ids intact, + `class: ""` (consistent with `teszt_enroll` and Part 1), mount+guest paths intact. +4. **storage.cfg clean**: zero `felhom-usb|felhom-flash` matches; `pvesm status` lists only + `local`, `local-lvm`, `felhom-pbs`, `felhom-offsite` — the untouchables all present and active. + +Post-op: controller `Up (healthy)` on 0.128.1; agent journal (last 10 min) free of +error/warn/detach — no false detach from the registry-sourced watchdog. ## Observations -- The per-drive target picker was deliberately left out (v1 = default drive only, per §12); if a - customer keeps exports on a non-default drive, the upload still lands on the default one and the - scan shows both — acceptable, but a picker is the natural follow-up. -- `controller/cmd/controller` is matched by a gitignore rule — `main.go` changes need - `git add -f` (it IS tracked; add refuses the path without `-f`). -- Startup GC logs only when it removes something; the clean deploy logged nothing (expected). +- The guest-bind mounts are `[/felhom-data]` subdir mounts (Model A) — a marker written at + `/felhom-data/` on the host appears at the guest drive ROOT (`/mnt/felhom-drives//`). + First marker check looked one level too deep; corrected before the removal. +- `felhom-flash` still carries empty PVE-era skeleton dirs on-disk (`images/ private/ snippets/ + template/ dump/`) from a past content=all era; harmless, deliberately NOT cleaned (task §12: no + touching drive contents). +- The `/etc/pve/.rrd` stats ring keeps historical series for the removed storages; it's + append-only telemetry and stops updating on its own — not a reference. +- The demo node now matches the fresh-install storage shape: no legacy dir: storages; drives are + registry+units-sourced only.