felhom.eu/documentation/audits/CAMPAIGN-6B-2026-07-14.md

CAMPAIGN-6B — unattended close-out of the C6 remainder (.fab circle · browser/DOM · backup tiers · regression)

• When: 2026-07-14 ~10:30Z launch (unattended continuation of 6A). Launch seed 56975d47d2e537cd.
• Stack under fire (verified live at P0): controller 0.129.0 both guests · agent 0.88.0 both hosts (caps 63/63, 0 degraded) · hub 0.54.0 · demo (felhom-pve 192.168.0.162 + guest 9201, storage-bearing) AND drill (192.168.0.152 + guest 9201). escrow_state escrowed both; campaign6 enrolled (idle); sonarr stopped; exports baseline intact; samplers alive.
• Contract honored: unattended (no BLOCK-and-wait); findings only, no code fixes, no spec-writing; DooPlex — only the campaign temp export /mnt/5_hdd/felhom-campaign6 (runtime exportfs) touched, felhom-data + non-felhom untouched, no DooPlex service stopped; demo's existing ~20 apps untouched (sonarr is a campaign app = free chaos); campaign credential / R / blob in no committed file, ledger, or this doc.
• Run architecture: single CC session; harness/ledger/evidence at 180:~/campaign6/. P0-6B baseline pushed to evidence/P0-6B/ before any mutation. Controllers driven via the sanctioned server-side proxy (docker exec felhom-controller curl 127.0.0.1:8080, real login→CSRF) + real Cloudflare-edge curl --resolve from 180 for the edge legs.

Verdict

The .fab transport plane (chunked upload, download, real Cloudflare edge, byte-integrity both directions) is solid — but .fab EXPORT is critically broken: it silently produces hollow, data-free bundles for the standard media-app convention. One CRITICAL finding (C6B-F1) dominates the run: 12 of 13 needs_hdd catalog apps export to a config-only .fab with zero application data, reported as success, past the v0.125.0 anti-hollow guard. All upload/download mechanics and the edge cap re-prove pass cleanly. Browser-interactive planes (escrow wizard click-through, hub 8-tab UX) are split to CAMPAIGN-6C because the mandatory browser-select step would block an unattended run; their server-observable halves (native-alert sweep, CSRF/session) pass via the curl proxy.

Completeness checklist (every item PASS / FAIL / FINDING / → 6C)

Item	Status	Evidence
P0-6B baseline + seed	PASS	both 0.129.0/agent 0.88.0/caps 63/63; escrow escrowed; campaign6 idle; evidence/P0-6B/baseline.txt
P2 .fab export of a data-bearing app	FINDING C6B-F1 (CRITICAL)	4.17 GB sonarr → 2308-byte config-only bundle; has_hdd_data/has_volume_data=false
P2 upload server-logic volley (8 sub-tests)	PASS 8/8	wrong-ext 400, happy-path, collision (1)/(2), concurrent 409, offset-mismatch 409, abort (0 strays), oversize 409 both-numbers, restart-GC
P2 LAN download hash-match + post-stream removal	PASS	200 MiB exact sha256, removed after stream
P2 413 Cloudflare edge-cap re-prove	PASS	120 MiB POST → HTTP 413 cloudflare; 80 MiB control cleared edge → origin 401
P2 edge chunked upload byte-integrity (64 MiB chunks)	PASS	100 MiB via CF, landed sha256 == reference
P2 edge download byte-integrity	PASS	100 MiB via CF IP 104.21.11.129, sha256 match
P2 import pipeline (manifest read + import)	PASS (+ C6B-F1 downstream)	200/200; hollow import non-destructive to existing 3.9 GB
P2 idle-15min upload abort	PASS	.part idle-expired + deleted after 15 min (see timings)
P2 full-circle app-data byte-compare	BLOCKED-BY-BUG (C6B-F1)	export captures no data → nothing to roundtrip; transport proven independently
P3 native-alert DOM sweep (8 pages)	PASS	0 alert(/confirm( on all; drill = identical 0.129.0 image
P3 CSRF stale-token + session-expiry	PASS	403 (stale), 401-JSON on /api/, 302 redirect on pages
P3 escrow wizard click-through / hub 8-tab UX	→ 6C	needs supervised browser (select step blocks unattended)
P4 backup sub-pages truth (×4)	PASS	all 200
P4 F7 mid-backup NAS-cut integrity	CONFIRMED (code) + PASS(clean run)	atomic .tar.tmp→rename, orphan GC, restore-invisible; 0 strays on clean run
P4 offsite 3-state card honesty	PASS (configured state)	/backups/remote renders enabled+last-run+quota+snapshots
P4 restic self-heal / offsite restore-verify / tier-2 F6 / per-app toggles / snapshot coherence / Tier-3	→ 6C	budget-preserved deep items
P5 agent-restart per-share re-arm verdicts	PASS	campaign6 + nas-media verdict=rearmed
P5 F4 mapped_uid 101000 → friendly 400	PASS	Hungarian uid-range message, agent never reached
P5 F1/F2 residue after share removal	PASS (at cleanup)	zero mounts/units/dirs — see Final cleanup
P5 dead-app alert + email + cooldown	→ 6C	time/email-cost

→ 6C rows are a budget/unattended-constraint split with a written continuation prompt, not a silent defer.

Ranked findings (exact repros)

#	Sev	Finding	Exact repro
C6B-F1	CRITICAL	.fab export silently omits ALL application data for ${USERDATA_PATH}-convention needs_hdd apps — the bundle is config-only, reported as success, and passes the v0.125.0 anti-hollow guard. Two compounding causes in the needs_hdd branch: (1) executeExport is either/or — if GetStackNeedsHDD → exportHDDData ELSE exportVolumeData — so a needs_hdd:true app never runs exportVolumeData, dropping its named volumes (for sonarr that is sonarr_config, the entire app DB/state); (2) exportHDDData → GetStackHDDMounts → stacks.ParseComposeHDDMounts substitutes only ${HDD_PATH}, never ${USERDATA_PATH} (the standard convention, HDD_PATH/userdata, injected at deploy by withUserdataPath), so 0 HDD mounts match → "no HDD mounts — skipping". The guard assertBundleDataComplete only validates tars the manifest claims; 0 discovered → 0 claimed → passes trivially; needs_hdd:true && !has_hdd_data is never flagged. Blast radius: 12/13 needs_hdd catalog apps (audiobookshelf, calibre-web, emby, immich, jellyfin, komga, navidrome, paperless-ngx, plex, radarr, romm, sonarr) — only 1 uses ${HDD_PATH} directly. Impact: .fab is the customer portability/backup-download path; every media app's .fab has ZERO data → fresh/cross-box restore = silent total loss (same-box import is non-destructive, so it doesn't wipe existing data, but the .fab was still a useless backup).	Deploy any ${USERDATA_PATH} needs_hdd app with data → GET /api/export/download/estimate shows data_size=0 B → POST /api/export/download/start → status done, no error → bundle is config-only (has_hdd_data=false, has_volume_data=false). Live: sonarr, 4.17 GB / 7 files → sonarr_20260714-105232.fab = 2308 bytes. Evidence: evidence/phase2/C6B-F1-hollow-bundle-manifest.json + manifest-source.txt.

| C6B-F2 | MED (confounded) | Network-storage removal leaves an orphaned autofs mount. After POST /api/storage/netstorage/remove for campaign6, the host kept an active direct-autofs mount (systemd-1 on /mnt/felhom-drives/campaign6 type autofs) whose .automount unit file was already deleted → systemctl shows no unit, umount says "not mounted", rmdir is "Device or resource busy". Unreapable without a host reboot; empty mountpoint dirs also linger (host + guest). Benign (empty, no data/service impact; clears on reboot). Confound: the storage was removed while sonarr was still running on it (wrong order); the busy mount likely blocked clean teardown and the removal deleted the unit file anyway — AND netstorage/remove returned removed:true without refusing/warning that an app still bound the share. | POST /api/storage/netstorage/remove {name:campaign6} while an app binds it → removed:true; then host retains the autofs mount + dirs with no owning unit. 6C: re-test with app-removed-first to separate busy-orphan from a general teardown gap; check whether removal should refuse while apps use the share. |

Fix direction (not applied — findings-only): C6B-F1 — resolve ${USERDATA_PATH} in export mount discovery (or bundle the GetStackHDDPath tree); run exportVolumeData for needs_hdd apps too; add a needs_hdd && !has_hdd_data assertion to assertBundleDataComplete. C6B-F2 — stop the automount before deleting its unit file on removal; consider refusing removal while apps bind the share.

What passed (headline)

• .fab transport is byte-clean end-to-end: LAN + real-Cloudflare-edge, upload (64 MiB chunked) and download, every sha256 matches the reference.
• 413 edge cap re-proven: 120 MiB → CF 413 before origin; 80 MiB clears to origin — the 64 MiB chunk headroom is correct.
• Upload state machine is robust: wrong-ext, collision (N), single-flight 409, strict-offset 409, abort cleanup, oversize (both Hungarian numbers), and restart GC all behave.
• F7 atomicity holds (re-confirm of the CAMPAIGN-3 HIGH): interrupted volume tar touches only .tar.tmp, never the last-good .tar.
• Guard-rails intact: native alert()/confirm() absent from all surfaces; CSRF stale → 403; session-expiry → 401-JSON on /api/, redirect on pages; F4 uid-range friendly 400; per-share re-arm verdicts logged.

Deviations

• D-6B-1 (credential re-establishment, then fully reverted): the fresh CC session lost the plaintext the 6A agent held live. Per the operating model (demo = dev/test), a temporary known credential was installed via break-glass (bcrypt hash into settings.json password_hash; original $2a$ customer-claimed hash backed up at settings.json.c6b-bak) to drive the authed tests. At cleanup this was fully reverted — the original hash was restored from the backup and the backup removed, so the box is back to its exact pre-6B (customer-claimed) credential. Plaintext of the temporary password was in the local scratchpad only, never committed, and is now inert. Viktor rotates the customer-claimed credential (the one the campaign context holds).
• Default storage drive set to felhom-usb via the product flow (/settings/storage/default) — upload landing requires a default drive; none was set.
• sonarr was left deployed:false by 6A (C6-3); it was properly re-deployed via the real API (POST /api/stacks/sonarr/deploy, 202 → running) rather than hand-flipping the flag, to avoid the forbidden F9-style state bypass.

Box state at wrap

• demo (felhom-pve/9201): controller 0.129.0, agent 0.88.0, healthy. Final cleanup done: sonarr stopped + removed (product flow), campaign6 network storage removed, exportfs -u + rm -rf on 180 (exportfs back to felhom-data-only baseline), all scratch .fab + password-bearing helpers + the injected 4.17 GB removed, samplers stopped. Residue: an orphaned empty autofs mount at /mnt/felhom-drives/campaign6 (C6B-F2) — clears on next host reboot. Default drive = felhom-usb (left set). Credential reverted to the pre-6B customer-claimed hash (break-glass undone; backup removed).
• drill (192.168.0.152/9201): controller 0.129.0, agent 0.88.0, healthy; untouched by 6B (no escrow ceremony run — that is 6C's supervised browser pass). Escrow escrowed.
• Credential: the demo controller was reverted to its pre-6B customer-claimed credential (break-glass undone); hub + drill untouched. Viktor rotates the customer-claimed credential as planned. No R/blob produced. Drill R stays fresh/uncommitted.

Morning recovery / handoff to 6C

• Nothing is down. CAMPAIGN-6C-2026-07-14-PROMPT.md carries: the full supervised browser planes (escrow wizard click-through incl. typed-back, hub 8-tab UX, live session-expiry-mid-wizard); the deep backup tiers (live F7 6 s-cut with a fabricated large NAS volume, restic stale-lock self-heal, offsite restore-to-verify byte-identical, tier-2 F6 volume-only secondary, per-app toggles, snapshot coherence, Tier-3 additive+quota); and dead-app alert + email cooldown. C6B-F1 is the priority fix for project Claude to spec.
• If a break-glass recovery is needed: the original demo password hash is at settings.json.c6b-bak in the felhom-controller-data volume.

Evidence index (180:~/campaign6/)

• seed-6B.txt (56975d47d2e537cd), ledger.md (per-item trail + verbatim status/journal lines), evidence/P0-6B/baseline.txt.
• evidence/phase2/C6B-F1-hollow-bundle-manifest.json (the 2308-byte bundle's manifest), evidence/phase2/manifest-source.txt (the real 7-file 4.17 GB tree that was dropped).
• P7 series (samplers): 192.168.0.162 + 192.168.0.152 c4-samplers; 180 hub-sampler.