@
CAMPAIGN-4: demo host reboot + drive cycle (operator-authorized risky steps) Demo HOST reboot PASS: 0 ordering cycles (even with active nas-media nfs4), retirement reboot-proof PASS (3 USB drives re-establish from agent units despite device-letter reshuffle), cards registry-sourced, nas-media nfs4 0 rbind/rc255 (F13 did not manifest). teszt_enroll drive cycle: eject/reattach work, wipe-gate + mount-guard + confirm-gate all correctly refuse unsafe format (no wipe); O-1/O-2 LOW observations. No credential/R/blob committed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LbMm4T7Ayzs1unB9pN6Uqd @
This commit is contained in:
@@ -17,11 +17,15 @@ A single unattended session cannot faithfully run the full 8h/two-box/nine-phase
|
||||
- **F-A (MED):** export size-estimate blind to docker-volume data — `download/estimate` reports `0 B`/"3.6 KB"/`fits_on_dest:true` while the real bundle is 1.2 GB; the pre-download honesty/free-space check is meaningless for volume-backed apps.
|
||||
- **F-C (LOW-MED):** `POST /api/escrow/claim` with no active ceremony returns HTTP 502 instead of a clean 4xx.
|
||||
|
||||
### Passed
|
||||
`.fab` export→download byte-integrity (ref==LAN sha256, 1.24 GiB, zero corruption); F12 reboot storm (0 cycles, 63/63 caps, full recovery ×5 boots); CSRF 403-JSON on missing token; escrow preflight honesty; P0 baseline both boxes; P8 samplers live.
|
||||
### Risky steps (operator-authorized mid-run, test env) — executed
|
||||
- **Demo HOST reboot — PASS (the load-bearing one):** host down ~29 s; **0 ordering cycles even with an active nas-media nfs4 mount**; caps 63/63; WG recovered. **P6 retirement reboot-proof PASSES** — all three USB drives re-established from agent units alone despite device letters reshuffling; `/api/disks` cards persist registry-sourced (no class badge); nas-media nfs4 re-established with **0 rbind/rc255** (F13 did not manifest). Drill VM restarted + recovered clean.
|
||||
- **teszt_enroll drive cycle — PASS:** eject/reattach work; the `.fab`/format **wipe-gate + mount-guard + customer-confirm gate** all correctly refuse an unsafe format (no accidental wipe). Two LOW observations (O-1 mount_path eject doesn't fully release the device; O-2 `assign` returns 200 without sync-restoring the mount). teszt data intact.
|
||||
|
||||
### Also passed
|
||||
`.fab` export→download byte-integrity (ref==LAN sha256, 1.24 GiB, zero corruption); F12 drill reboot storm (0 cycles, 63/63, ×5 boots); CSRF 403-JSON; escrow preflight honesty; P0 baseline both boxes; P8 samplers live.
|
||||
|
||||
### Deferred (ready procedures in the audit doc)
|
||||
Demo host reboot (retirement reboot-proof; F12 already proven on identical-version drill), `.fab` upload/import + CF-edge legs (need a storage-bearing box), NAS regression ring, backups tiers, escrow wizard + UI DOM sweeps + hub 8-tab ring (browser available; deferred for runway).
|
||||
`.fab` upload/import + CF-edge legs (need a storage-bearing box), NAS regression ring, backups tiers, escrow wizard + UI DOM sweeps + hub 8-tab ring (browser available; deferred for runway).
|
||||
|
||||
### Box state
|
||||
Nothing down. demo untouched (host not rebooted, apps intact, escrow read-only/paper-R valid). drill: 5 boots, healthy, scratch `c4_integrity/` + `/root/ref1.fab` (safe to delete); escrow reset to `phase:none` (transient) — needs a fresh ceremony if a paper R is wanted. **Campaign credential active on both controllers — Viktor rotates in the morning.** Evidence at `180:~/campaign4/`.
|
||||
Nothing down; both hosts + guests healthy after reboots. demo host rebooted + recovered; existing apps untouched; `teszt_enroll` cycled (data intact, `/mnt/teszt_enroll` mount pending agent-reconcile restore). drill: 6 boots, healthy, scratch `c4_integrity/` + `/root/ref1.fab` (safe to delete); escrow `phase:none` (transient) — needs a fresh ceremony for a paper R. **Campaign credential active on both controllers — rotate in the morning.** Samplers still running (stop: `pkill -f c4-sampler.sh` on both PVE hosts, `pkill -f hub-sampler.sh` on 180). Evidence at `180:~/campaign4/`.
|
||||
|
||||
@@ -7,14 +7,14 @@
|
||||
|
||||
## Status of this run
|
||||
|
||||
This is a **partial campaign with a high-fidelity evidenced core**, honestly scoped. A single unattended agent session cannot faithfully execute the full 8-hour, two-box, nine-phase destructive matrix (4 GiB round-trips + reboot storms + NAS chaos + escrow wizard + backups tiers + hub UI ring) to the depth its value requires **without fabricating results** — which would defeat a "no mercy" campaign whose worth is real findings with exact repros. So the priority items were executed for real with evidence; the rest is **explicitly deferred with reasons and ready procedures** (see Deferred scope). Nothing here is claimed that was not observed. The endurance samplers run all night regardless.
|
||||
High-fidelity evidenced core, honestly scoped; nothing here is claimed that was not observed; the endurance samplers run all night. **Mid-run update:** after the first active block the operator explicitly authorized the risky steps to run unattended (test environment). So the two biggest "deferred" items — the **demo HOST reboot** (F12-on-real-box + P6 retirement reboot-proof) and the **teszt_enroll drive cycle** — were then executed and are reported below. Remaining phases (NAS regression ring, `.fab` upload/import leg, backups tiers, browser wizard/UI/hub-tab ring) stay deferred with ready procedures rather than fabricated.
|
||||
|
||||
## Verdict
|
||||
|
||||
**The data-movement and reboot-recovery planes are solid on the code paths exercised; three real product findings surfaced, one of them a genuine auth-hardening gap.**
|
||||
The `.fab` export→download integrity leg is byte-perfect. The **F12 reboot-safety fix HOLDS** — the exact CRITICAL that ended CAMPAIGN-3 is closed: a drill-host reboot storm (5 boots) produced **zero ordering cycles**, full networking/WG/agent/capabilities(63/63)/guest/controller recovery every time. Three findings became the take-aways: a login rate-limiter keying bug (MED, red-proofed), an export size-estimate blind to volume data (MED), and an ungraceful 502 on a no-op escrow claim (LOW-MED).
|
||||
The `.fab` export→download integrity leg is byte-perfect. The **F12 reboot-safety fix HOLDS on both boxes** — the exact CRITICAL that ended CAMPAIGN-3 is closed: a drill-host reboot storm (5 boots) AND a **demo-HOST reboot** (with an active nas-media nfs4 mount present) both produced **zero ordering cycles** with full networking/WG/agent/capabilities(63/63)/guest/controller recovery. The **P6 retirement reboot-proof PASSES**: all three USB drives re-established from agent units alone despite their device letters reshuffling across the reboot. The `.fab` **wipe-gate + mount-guard + customer-confirm gate** all correctly refuse an unsafe format. Three findings became the take-aways: a login rate-limiter keying bug (MED, red-proofed), an export size-estimate blind to volume data (MED), and an ungraceful 502 on a no-op escrow claim (LOW-MED).
|
||||
|
||||
Ledger (active block): **6 PASS · 3 FINDING · 1 OBSERVATION · 4 DEVIATION/INFO · 6 DEFERRED-phase**.
|
||||
Ledger: **9 PASS · 3 FINDING · 3 OBSERVATION · 4 DEVIATION/INFO · 4 DEFERRED-phase**.
|
||||
|
||||
## Top findings (ranked — these become the next specs)
|
||||
|
||||
@@ -28,6 +28,8 @@ Ledger (active block): **6 PASS · 3 FINDING · 1 OBSERVATION · 4 DEVIATION/INF
|
||||
|
||||
- **`.fab` export→download integrity (drill, PASS):** `stop_app` export of `actualbudget` with a >1 GiB injected dataset → **1.2 GB** bundle (steps Előkészítés→Titkosítás all green). `docker cp` reference sha256 `532fd003…d637` == LAN endpoint-download sha256, **1,328,973,695 bytes, zero corruption**. Confirms the v0.125 docker-cp streaming volume leg captures data correctly (contra the F-A estimate).
|
||||
- **F12 reboot-safety / DR-plumbing recovery (drill storm, PASS — the CAMPAIGN-3-killer, now closed):** 5 total boots. Every genuinely-observed post-boot state: **ordering cycles = 0**, agent active 0.88.0, **capabilities ok=63 total=63 degraded=0 inactive=0**, WG `wg-felhom` re-handshake <30 s, guest 9201 autostart, controller "healthy" within ~50 s (SSH-up ~46 s). Only benign `binfmt_misc` automount lines in the journal — zero felhom ordering cycles. The F12 fix holds and is repeatable.
|
||||
- **DEMO HOST reboot — F12-on-real-box + P6 retirement reboot-proof (PASS, the load-bearing one):** reboot issued 22:32:40Z, host down **~29 s**. **0 ordering cycles even with an active `nas-media` nfs4 mount** (`192.168.0.180:/mnt/5_hdd/felhom-data`) present pre-reboot; caps 63/63; WG recovered. **All three USB drives (teszt_enroll / felhom-flash / felhom-usb) re-mounted + guest-bound from agent units alone despite their device letters reshuffling** (teszt sdb→sdc, flash sdd1→sdb1, usb sdc1→sdd1) — the durable-id/registry model absorbed it. `/api/disks` cards persisted registry-sourced (durable_id present, `class:""` = v0.128.1 badge-suppression holds), correct roles. `nas-media` nfs4 re-established with **0 rbind/rc255 errors** (the open **F13** "active nfs4 → rbind rc255" did **not** manifest this reboot). Guest + controller healthy; both samplers relaunched; drill VM (onboot=0) restarted manually and recovered clean (0 cycles).
|
||||
- **`.fab` wipe-gate + mount-guard + customer-confirm gate (demo teszt_enroll, PASS):** `POST /api/disks/format` refuses a **mounted** device (`reason:"device is mounted"`) regardless of `confirmed`; unconfirmed user-data → `needs_confirmation:true`. No accidental wipe reachable. Eject (`where=/mnt/teszt_enroll`) and reattach (`assign`) both 200. teszt_enroll data left intact.
|
||||
- **CSRF protection (drill, PASS, edge-safe):** `POST /api/escrow/claim` with no `X-CSRF-Token` → **403 JSON** `{"ok":false,"error":"CSRF token missing or invalid"}` (no native alert, JSON on `/api/`).
|
||||
- **Escrow preflight honesty (drill, PASS):** `escrow_state:"escrowed"`, items `pbs_storage_id:felhom-pbs`, `dr_tier:applied`, `age_binary:/usr/bin/age`, `hub_upload:configured` all `ok:true`; `staged_secret:false` correctly flagged informational.
|
||||
- **P0 baseline captured** for both boxes (versions, disks, escrow, backup, apps) — the diff anchor.
|
||||
@@ -38,6 +40,8 @@ Ledger (active block): **6 PASS · 3 FINDING · 1 OBSERVATION · 4 DEVIATION/INF
|
||||
- **Escrow status is transient across reboot.** Post-reboot `/api/escrow/status` → `phase:none, claimed:false` (was `done/claimed/uploaded fp f2:87…` pre-reboot); hub-side truth persists (preflight still `escrowed`). The local ceremony job state is in-memory and resets on restart — consistent with the campaign's own "restart → status honestly none" expectation. Note the coupling with F-C: a claim in this state 502s instead of returning a clean state.
|
||||
- **DELETE guard works:** `DELETE /api/stacks/actualbudget` on a deployed app → refused `"not orphaned — only orphaned stacks can be deleted"`.
|
||||
- **`.fab` upload requires a default storage drive:** on the drill box (no user-data drive) `POST /api/export/upload/init` → honest Hungarian `"Nincs alapértelmezett tároló beállítva."` — correct behavior, but it means the upload/import leg cannot run on a storage-less box.
|
||||
- **O-1 (LOW): a mount_path eject does not fully release the device.** After `POST /api/disks/eject {where:"/mnt/teszt_enroll"}` (200), the device stays mounted via the `/mnt/felhom-drives/teszt_enroll` namespace bind, so a subsequent `format` correctly still refuses (`"device is mounted"`). A format-intent flow that ejects only the mount_path can never reach a wipe. Whether the UI's format flow detaches all binds first is untested. Also: eject `where` **must** be the mount_path — the bare name / guest_path / `uuid:` all 403 with a `role: system` misclassification (they resolve to the `/mnt/felhom-drives` parent on pve-root).
|
||||
- **O-2 (LOW): `POST /api/disks/assign` returns 200 but did not synchronously restore the mount.** After a successful re-attach the `/mnt/teszt_enroll` host mount stayed absent (guest bind + data intact); the agent reconcile / next reboot restores it (reboot-restore proven above).
|
||||
|
||||
## Deviations ledger
|
||||
|
||||
@@ -49,17 +53,18 @@ Ledger (active block): **6 PASS · 3 FINDING · 1 OBSERVATION · 4 DEVIATION/INF
|
||||
|
||||
## Deferred scope (reasons + ready procedures — for a supervised continuation)
|
||||
|
||||
1. **Demo HOST reboot (P1-F12 on the real box + P6 retirement reboot-proof)** — the load-bearing "felhom-usb/felhom-flash re-establish from agent units alone after their PVE-dir retirement" test. **Deferred from this active block** per the ordering's T+3h evidence-banking gate and because rebooting Viktor's demo Proxmox host unattended is the single riskiest action here. F12 is already proven on the identical-version drill host. Procedure: capture `/api/disks` + `journalctl -b0|grep -c 'ordering cycle'` + agent caps → `reboot` felhom-pve → verify 0 cycles, both USB mounts + guest binds re-establish, controller cards registry-sourced (no class badge), caps 63/63, WG/pbs markers.
|
||||
2. **P2 upload→import→byte-compare + CF-edge legs** — the export/download half PASSED on drill; the upload half needs a default storage drive (drill has none) and the CF-edge `--resolve` 413 re-proof needs the box's public hostname. Best run as the **demo additive pass** (new small campaign app on the storage-bearing demo box) to avoid colliding with existing demo apps.
|
||||
3. **P1 NAS regression ring (F4–F12 NAS specifics)** — requires enrolling a DooPlex NFS share and the full unexport/idle/reboot matrix. Not started this block.
|
||||
4. **P5 backups IA + tiers depth, P6 teszt_enroll drive cycles** — not started.
|
||||
5. **P3 escrow wizard via browser + P4 UI DOM sweeps + P7b hub 8-tab ring** — browser IS available; deferred for runway. The curl-equivalents of P3/P4 (CSRF, rate-limiter, escrow status/preflight/claim) WERE run (above).
|
||||
6. **P7a hub server-side verification sweep** — hub pod health sampled (P8); per-act `host_escrow` row diffing not run.
|
||||
1. ~~Demo HOST reboot~~ **DONE** (operator-authorized mid-run) — PASS, see What passed.
|
||||
2. ~~teszt_enroll drive cycle~~ **DONE** (operator-authorized mid-run) — eject/reattach + wipe-gate PASS, see What passed + O-1/O-2.
|
||||
3. **P2 upload→import→byte-compare + CF-edge legs** — the export/download half PASSED on drill; the upload half needs a default storage drive (drill has none) and the CF-edge `--resolve` 413 re-proof needs the box's public hostname. Best run as the **demo additive pass** (new small campaign app on the storage-bearing demo box) to avoid colliding with existing demo apps.
|
||||
4. **P1 NAS regression ring (F4–F12 NAS specifics)** — requires enrolling a DooPlex NFS share and the full unexport/idle/reboot matrix. Not started.
|
||||
5. **P5 backups IA + tiers depth** — not started.
|
||||
6. **P3 escrow wizard via browser + P4 UI DOM sweeps + P7b hub 8-tab ring** — browser IS available; deferred for runway. The curl-equivalents of P3/P4 (CSRF, rate-limiter, escrow status/preflight/claim) WERE run (above).
|
||||
7. **P7a hub server-side verification sweep** — hub pod health sampled (P8); per-act `host_escrow` row diffing not run.
|
||||
|
||||
## Box state at wrap
|
||||
|
||||
- **demo (felhom-pve / guest 9201):** controller 0.128.1, agent 0.88.0, all apps as found (untouched), escrow `fp b0:fe:2a…` done/sealed (read-only, paper R valid). Disks unchanged (`teszt_enroll` sacrificial stick intact, not yet cycled). Host **not rebooted**. P8 `demo` sampler running (`/root/c4-demo.csv`).
|
||||
- **drill (192.168.0.152 / guest 9201):** controller 0.128.0, agent 0.88.0, 5 boots tonight, healthy. **`actualbudget` was DELETE-attempted (refused, still present) and had a `c4_integrity/` folder (~1.3 GiB) injected into its volume** — scratch data, safe to delete; app itself intact. A scratch `.fab` (`/root/ref1.fab`, 1.24 GiB) sits on the drill host — delete at will. Escrow now `phase:none` (transient reset by reboot; hub-side still escrowed). **Drill needs a fresh Viktor-run ceremony if a paper R is wanted for it** (per contract §4). P8 `drill` sampler running (`/root/c4-drill.csv`).
|
||||
- **demo (felhom-pve / guest 9201):** controller 0.128.1, agent 0.88.0, all apps as found (untouched), escrow `fp b0:fe:2a…` done/sealed (read-only, paper R valid). **Host WAS rebooted (authorized) and recovered clean** (0 cycles, all drives + binds re-established, caps 63/63). **`teszt_enroll` was eject/reattach-cycled; its data is intact (NOT wiped)** but its `/mnt/teszt_enroll` host mount is currently unmounted (guest bind + data intact) — the agent reconcile / a controller restart / next reboot restores it (reboot-restore proven tonight). P8 `demo` sampler relaunched (`/root/c4-demo.csv`).
|
||||
- **drill (192.168.0.152 / guest 9201):** controller 0.128.0, agent 0.88.0, 6 boots tonight (storm + the demo-host reboot took it down; restarted via `qm start 300`, onboot=0), healthy. **`actualbudget` had a `c4_integrity/` folder (~1.3 GiB) injected into its volume** — scratch data, safe to delete; app intact (a DELETE was refused — not orphaned). A scratch `.fab` (`/root/ref1.fab`, 1.24 GiB) sits on the drill host — delete at will. Escrow now `phase:none` (transient reset by reboot; hub-side still escrowed). **Drill needs a fresh operator-run ceremony if a paper R is wanted for it** (per contract §4). P8 `drill` sampler relaunched (`/root/c4-drill.csv`).
|
||||
- **Credential:** campaign credential active on both controllers — **Viktor rotates in the morning**.
|
||||
|
||||
## Morning recovery / cleanup
|
||||
|
||||
Reference in New Issue
Block a user