diff --git a/CONTEXT.md b/CONTEXT.md index b996362..300d8f8 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -7,7 +7,21 @@ > > Ask Claude Code: "Please update CONTEXT.md with what we did today" -Last updated: 2026-07-16 (v0.140.0 — Direction-2 immediate-sync wait channel; the immediate-sync arc is COMPLETE for the controller plane) +Last updated: 2026-07-17 (v0.141.0 — N100 polish: F6 initialize-to-usable + F7 back-routes) + +> **2026-07-17 — v0.141.0: N100 polish (F6 + F7), LIVE on demo.** F6 (MEDIUM): drive "initialize" +> now ends in a mounted+registered drive even on a client disconnect. `POST /api/storage/init` runs +> the format→mount→register chain as a DETACHED single-flight job (`web/storage_init_job.go`, +> `context.Background()`, netAddState shape) the wizard polls via `GET /api/storage/init/status` +> (3-step Hungarian progress); register is the last step (marker-last crash-safety). Fork verdict: +> controller-side, NO agent change (the chain must reach FileBrowser sync = controller-only). +> **Deeper half found on the live leg:** a slow mkfs (64 GB USB, ~27 s) outruns the agentapi client's +> 15 s timeout → the controller now polls the agent's `GET /disks/format/status` +> (`agentapi.FormatStatus` → `awaitAgentFormat`) then continues. Live-validated on `/dev/sdd` → +> `/mnt/felhom-drives/scratch1` (mounted+registered). F7 (LOW): storage init/attach Vissza → `/storage`. +> Red-proofs for both F6 halves + F7. Security review of the commit flagged the pre-existing +> format→resolve→assign device-node TOCTOU (agent-guarded destructive step, benign fs-UUID mount) — +> acknowledged as an Observation, not expanded. Fork/landmarks/live evidence: REPORT.md. > **2026-07-16 — v0.140.0: Direction-2 immediate-sync (hub→box) SHIPPED.** The reverse of v0.139.0: > an OPERATOR action on the hub now reaches the box in seconds. `report.Waiter` diff --git a/REPORT.md b/REPORT.md index 961adf2..4d3dd21 100644 --- a/REPORT.md +++ b/REPORT.md @@ -1,56 +1,71 @@ # REPORT — most recent implementation -## v0.140.0 — Direction-2 immediate-sync: hub→box wait channel client (2026-07-16) +## v0.141.0 — N100 polish: initialize-to-usable (F6) + Vissza back-routes (F7) — 2026-07-17 -The reverse half of the immediacy arc (Direction 1 = v0.139.0 box→hub trigger). An operator action -on the hub now reaches the box in **seconds** instead of on the next ~15-min cycle. Pairs with hub -**v0.58.0** (`GET /api/v1/wait` + the in-memory operator-intent notifier). Grounding: -`felhom.eu/documentation/audits/SPIKE-immediate-sync-transport-2026-07-16.md` (option b). +**Baseline:** v0.140.0 (`5be2449`) → **v0.141.0** (deployed live on demo 9201, healthy). Green: +`go build ./... && go vet ./... && go test ./...` all pass; template/emoji/native-confirm gates pass. +Closes `felhom.eu/documentation/tests/VALIDATION-n100-baremetal-2026-07-16.md` F6 + F7. (F5 lives in +felhom.eu `scripts/`; see that repo's REPORT.) -### What shipped -- **`internal/report/waiter.go` (new) `report.Waiter`.** Holds a hanging authenticated - `GET {hub}/api/v1/wait?gen=N` (same hub URL + key as the pusher — **no new config keys**). Its own - `http.Client` has **no overall Timeout** (a held GET must stay open for the hub's ~240 s hold), - with connect/TLS/`ResponseHeaderTimeout` deadlines; a per-request context bounds a black-holed - connection. On a completion whose generation **differs** from the last seen, it fires the v0.139.0 - `report.Trigger` — and nothing else; the report ACK delivers config/escrow/claim/floor through the - UNCHANGED machinery (zero new delivery logic; the box pulls even the wake-up). - - First observation **records, never fires** (no restart-echo report). - - Same-generation timeout **fires nothing** (not interval-shortening). - - Heartbeat newlines tolerated; only the `{"gen":N}` line is read (contentless wake). - - Any error — transport, **404 from a pre-v0.58.0 hub**, malformed body — backs off 5 s→5 min - (reset on success), ONE WARN per state change, and the 15-min cycle keeps reconciling. Exits - promptly on ctx cancel (even mid-hold). -- **`cmd/controller/main.go`.** Waiter constructed + started beside the Direction-1 trigger, gated on - the SAME `hubPusher != nil && cfg.Hub.Enabled` (strict no-op when hub reporting is off). -- **Copy soften.** `backups_remote.html` + `backups_escrow.html`: "ez általában néhány **másodperc**, - legfeljebb 15 perc" (the 15-min bound stays as the honest worst case; escrow grace window - unchanged). +### Fork verdict (Part 1 Phase 1, source-grounded) +**F6 is controller-side — NO agent change.** The detached mkfs lives agent-side +(`felhom-agent/internal/localapi/formatjob.go:99` + `disks.go:635`, crash-safe, mkfs-only), but the +format→mount→register ORCHESTRATION is `internal/web/storage_handlers.go runStorageInit` and ran on +the REQUEST context; a disconnect cancelled it after `FormatDisk` (agent mkfs continues detached, +returns `errFormatClientGone`) → the mount+register leg never ran → device formatted but unusable. +The chain must reach `SyncFileBrowserMounts` (`handlers.go:2011`, controller-only), so a single +detached job can only live controller-side. The agent already exposes crash-safe format + assign → +agent v0.90.0 was NOT triggered. -### Tests + red-proof -Non-hollow `waiter_test.go` (httptest fake hub): fires-once-on-change, same-gen-timeout-no-fire, -**first-observation-records-no-fire** (RED-PROOF: disable the baseline branch → first poll fires; -run-fail-reverted), heartbeats-tolerated, malformed-completion-no-fire-no-panic, 404-as-error, -Bearer+gen wire check, ctx-cancel-mid-hold-returns-promptly. Full controller suite green -(`go build/vet/test ./...`); design gates pass (template_id / emoji / native_confirm / offbox_rename -/ mojibake). +### F6 — initialize now ends in a mounted+registered drive, disconnect-safe +- `POST /api/storage/init` starts a DETACHED single-flight job (`internal/web/storage_init_job.go`, + the `netAddState` shape) on `context.Background()`; `runStorageInit` gained a nil-safe phase callback + (formatting → mounting → registering). The wizard polls `GET /api/storage/init/status` and renders + the 3-step progress (`storage_init.html`); confirm/refuse verdicts surface through the same poll. + Register is the LAST step (marker-last, Scenario B) + idempotent prior steps (`AddStoragePath` + dedups) → a crash leaves at most an unregistered orphan, never a broken/duplicate registration. +- **Deeper half (found on the live leg):** a slow mkfs (a 64 GB USB) outruns the agentapi client's + 15 s `Timeout`; the agent runs it DETACHED and records the job, so `runStorageInit` now POLLS the + agent's `GET /disks/format/status` (new `agentapi.Client.FormatStatus` → `awaitAgentFormat`) to the + terminal outcome on a client timeout, then continues to mount+register (the F6 root-cause's "mkfs + continues detached; poll the status"). +- **Red-proofs (run-fail-revert):** `TestStorageInit_DetachedSurvivesClientDisconnect` (pre-fix + cancelled-ctx chain fails at mount, NOT registered → FAIL; fixed: detached job registers exactly + once) + `TestStorageInit_PollsAgentFormatStatusOnTimeout` (timeout→done registers; timeout→failed + surfaces the error). -### Deploy + live validation (guest 9201, customer demo-felhom) -Image `felhom-controller:0.140.0` built + pushed on 180; deployed via the bootstrap flow; container -`Up (healthy)`; log `[report] hub wait channel active (hold ≤240s)`; startup report sent. -- Completion-on-difference path: box-side curl with a mismatched `?gen=999999` → `{"gen":0}` in - **0.047 s** through the real ingress. -- No-annotation hold proof: box-side curl at `?gen=0` held cleanly for **240.04 s** (HTTP 200 → - `{"gen":0}`, **10 heartbeat newlines**); the controller's own Waiter logged **no wait-channel - errors** over the window (holds + reconnects cleanly past nginx's 60 s default). No ingress - annotation applied. -- Scenario C (hub `rollout restart` mid-hold): the Waiter logged **exactly one** WARN (`unexpected - EOF — backing off`), reconnected cleanly, and fired **0 out-of-cycle reports** (gen 0→0, no storm). - The "recovered" INFO is deferred to the next hold completion (`pollOnce` blocks) — cosmetic. -- Save→applied via the operator UI needs an operator login (CC lacks the password by standing - constraint); the mechanism is unit-proven and the delivery chain is the already-live v0.139.0 / - config-refresh path. +### F7 — Vissza back-routes +The "Vissza" (Back) anchor on `/storage/init` and `/storage/attach` now routes to `/storage` (was +`/settings`); the init success link also points to `/storage`. Test +`TestStorageWizardBackAnchors_PointToStorage`. -### Coupling -Immediacy needs hub ≥ v0.58.0; against an older hub the wait 404s and the box degrades to the 15-min -cycle. No agent coupling, no `MinAgent`. The agent-plane ep0-poke stays parked in the OOB arc. +### Live validation (F6 Scenario A, on the demo scratch USB) +Method: the REAL controller endpoints (curl against the in-guest controller with the dashboard Host +header + a login session + the CSRF token — the exact calls the UI makes; browser tool not used). +Scratch device `/dev/sdd` (serial `20120926571200000`, 59.5 GB) serial-checked against the enrolled +list FIRST (≠ the enrolled `/dev/sdb` hdd_1; ≠ OS `/dev/sda`) and confirmed present in the +agent-proven `initialize` candidates. Flow: init (no confirm) → `needs_confirmation` (ext4) via poll → +init (confirmed) → `{started}` (returns immediately; the client can leave) → detached job +`formatting` (~27 s mkfs, past the 15 s client timeout → the status-poll fix engaged) → `mounting` → +**`done`** at `/mnt/felhom-drives/scratch1`. Verified: `/dev/sdd` mounted at the enrolled stable bind +`/mnt/felhom-drives/scratch1` (`/dev/sdd[/felhom-data]`); the enrolled hdd_1 untouched. + +### Security-review acknowledgment (background review of the commit) +Flagged a TOCTOU in `runStorageInit` (format → `resolveEnrollUUID(path)` → `AssignDisk(uuid)`). +**Assessment:** pre-existing (unchanged sequence — F6 only moved it off the request context and added +the timeout status-poll, which widened the window by the poll interval). The DESTRUCTIVE step (format) +is the authorization boundary and is agent-enforced with anti-retarget **durable-id** binding — the +controller cannot make the agent format the wrong device; the subsequent mount is benign and keyed by +**fs-UUID** (a moved /dev node resolves to a non-existent UUID → a mount failure, not data loss or an +authz bypass). No new authorization bypass. Left as an Observation (not expanded this batch per the +minimal-changes mandate); a future hardening could bind resolve+assign to the format's durable-id. + +### Operator sidebar (Viktor — CC does NOT do these; recorded verbatim from the spec) +1. **BIOS "State After G3" → "Power On"/"Last State"** before final sign-off — also added to the + hardware-prep notes in `scripts/iso/README.md`. +2. **campaign6 autofs orphan** — clears at the next host reboot. +3. **tied-CreatedAt flash duplicates** (audiobookshelf/komga/romm) — standing action item. + +### NOT live this task +F5 (guest auto-sizing) is validated by harness only; live at the next from-scratch rehearsal +(Viktor's planned delete/re-create run) — see felhom.eu REPORT. diff --git a/controller/README.md b/controller/README.md index 946b0f9..82e44fa 100644 --- a/controller/README.md +++ b/controller/README.md @@ -1081,11 +1081,19 @@ not just those with HDD data. Non-HDD apps can configure destination, method, an > name is typed exactly**. Wipe (`POST /api/storage/wipe`): eject (unmount + deregister) → server-side > two-step customer-confirmed format (learn the agent's durable id, then re-submit `confirmed:true` bound > to it). The agent refuses a protected device regardless of what the controller sends. -> - **Guided init** (`/settings/storage/init`, `POST /api/storage/init`, `web/storage_handlers.go`): format +> - **Guided init** (`/storage/init`, `POST /api/storage/init`, `web/storage_handlers.go`): format > → resolve the new fs UUID → `assign` → register. The selector lists **only user-data** targets. A > data-bearing user-data device now uses the **customer-confirmation** flow (type-to-confirm → re-submit > `confirmed:true` + durable id), NOT the `felhom-opsign` command. The opsign surface remains a fallback > only if a protected device somehow reaches init. +> **F6 (v0.141.0):** the chain runs as a DETACHED single-flight job (`web/storage_init_job.go`, +> `context.Background()`) the wizard polls via `GET /api/storage/init/status` (3-step progress: +> formatting → mounting → registering) — a closed tab / lost connection no longer aborts the +> post-mkfs mount+register. `register` is the LAST step (marker-last crash-safety). A slow mkfs that +> outruns the agentapi client's 15 s timeout is followed by polling the agent's +> `GET /disks/format/status` (`agentapi.Client.FormatStatus` → `awaitAgentFormat`) before +> continuing. Live-validated on a 64 GB USB (mkfs ~27 s → done, mounted+registered at +> `/mnt/felhom-drives/scratch1`). > - **Guided attach** (`/settings/storage/attach`, `POST /api/storage/attach`): non-destructive — resolve > the existing fs UUID → `assign` → register. Selector restyled to cards (user-data only). > - **Eject** (`POST /api/storage/eject`): benign unmount + deregister, with the agent's dependent-guest