felhom.eu/documentation/audits/CAMPAIGN-6C-2026-07-14.md

CAMPAIGN-6C — supervised planes + deep tiers + v0.130–0.132 fix-verifies (2026-07-14)

Seed: 6C-20260714 · Ledger/evidence: 180:~/campaign6/ (6C-P0.md + 6B ledger.md/evidence) Continuation of: CAMPAIGN-6B. Class: acceptance + findings. Boxes: controller v0.132.0 both guests · agent 0.88.0 · hub 0.54.0.

This pass ran unattended (operator-approved) via the controller's own API — authenticated with the Campaign-4 credential (out-of-band, 180:~/.config/credentials), driving the exact endpoints the dashboard invokes (docker exec felhom-controller curl 127.0.0.1:8080 + session cookie + CSRF). This is the live-test-real-flow acceptable proxy where a rendered browser wasn't used. No credential, R, or blob is written here. Cookie/CSRF were transient (container /tmp, wiped at teardown).

---

Checklist

Bucket	Leg	Result
P0	baselines (ctrl 0.132.0 / agent 0.88.0 / hub 0.54.0), backup block rejected=0, storage headroom	PASS
P0	API access mechanism (session-cookie auth, no loopback bypass)	CHARACTERIZED
4P-1	deploy paperless-ngx (felhom-usb), ingest 3 varied docs via consume flow	PASS
4P-2	F-S2 tier-2: appdata leg mirrors appdata/paperless byte-identical	PASS (+ F-6C-1)
4P-3	F-S2 marker restore: resolved live dir, additive-only, byte-identical	PASS
4P-4	storage page shows paperless-ngx non-empty size (39.6 KB)	PASS
4P-5	F-S3 scope="app" migration usb→flash→usb, full manifest, no ghost dir, no WARN	PASS
5-rest	dead-app alert	INCONCLUSIVE (wrong trigger — clean stop ≠ crash)
Accept #1	C6B-F1 .fab ≥1 GiB full-circle byte-compare	DEFERRED (needs dedicated setup)
PHASE 3	escrow wizard / session-CSRF UX / hub 8-tab ring (browser)	NOT-RUN (needs supervised browser)
PHASE 4-deep	F7 live cut, restic self-heal, offsite restore-verify, snapshot coherence	NOT-RUN (deep-setup budget)

---

Verdict

GREEN for the v0.131.0 acceptance. All five PHASE 4-paperless legs PASS — this fully closes the "NOT live-validated" list in the v0.131.0 REPORT (F-S2 tier-2 backup, F-S2 marker restore, F-S2 storage-page size, F-S3 scope="app" migration round-trip), on the live demo through the real dashboard endpoints, with byte-identical manifests at every step and no appdata/paperless-ngx ghost dir and no declared appdata dir absent WARN. One new MEDIUM finding (F-6C-1, tier-2 to an NFS target). The browser planes (PHASE 3) and deep tiers (PHASE 4-deep) + the .fab ≥1 GiB circle remain for a supervised/deep-budget pass (6D) — the credential-gated automation ran, but those legs need a rendered browser (operator-typed password + on-screen typed-back) or large-dataset/timed-cut setup this unattended pass did not provision.

---

Findings (ranked)

F-6C-1 (MEDIUM) — tier-2 auto-target = NFS share fails at the recovery-unit leg (rsync -a chown)

RunAllTier2 auto-picked nas-media (NFS) as paperless-ngx's off-drive target. rsyncMirror (internal/backup/tier2.go) uses rsync -a, whose -o -g preserve owner/group → chown on the NFS export fails ("Operation not permitted", root_squash) → exit 23, the recovery-unit leg fails first, so RunTier2 records failure and the appdata leg never runs. Same class hit calibre-web (teszt_enroll mkdir permission-denied — a separate bad-mount residue).

• Repro: any box whose first schedulable off-disk tier-2 target is an NFS share.
• Impact: tier-2 silently produces no off-drive copy for the affected app (status = failure, so it's honest, not silent — but no protection). Real drives (felhom-usb/flash) are unaffected.
• Not an F-S2 regression — F-S2 (the appdata dir-name resolution) is proven correct once the target is a real drive (4P-2 byte-compare).
• Suggested fix (→6D / a controller task): for a network-fs target, drop -o/-g (add --no-owner --no-group, or --chown), or exclude NFS mounts from the auto tier-2 target pick.
• Evidence: controller log 2026-07-14 17:35:31 (exit 23 chown lines); 180:~/campaign6/6C-P0.md.

Observation (LOW, → 6D) — aggregate state "running" while main container Exited(0)

A clean docker stop paperless-webserver (exit 0) did NOT fire a dead-app alert in 2.5 min and the controller reported state=running / health-probes "4 ok". This is likely by design (a clean stop is not a crash; downstate_test asserts certain states must not manufacture an alert), so 5-rest is inconclusive with this trigger. Worth a 6D note: the aggregate state reads "running" while the service container is down (other containers up). A proper crash trigger (docker kill -9 / OOM, exit≠0) is the correct test — deferred (restart-policy race needs care).

---

Timings (paperless acceptance, demo)

Step	Wall time
deploy → 3 containers healthy	~95 s (Postgres start_period)
3-doc consume ingest (OCR + PDF/A)	~90 s
tier-2 run (all HDD apps)	~30 s
tier-2 file-restore (stop→copy→start→health)	39 s
migration usb→flash (scope=app)	45 s
migration flash→usb (round-trip)	45 s

---

Deviations from the prompt

• PHASE 3 / 4-deep / Accept #1 not run. PHASE 3 needs a rendered browser with an operator present (browser-select prompt, password typed at the login screen, and the wizard's "type the two highlighted words read from screen") — genuinely interactive, not unattended-automatable. 4-deep (F7 live 6 s cut, restic mid-run kill, offsite restore-to-verify) and Accept #1 (≥1 GiB userdata app
  • encrypted .fab round-trip) need large-dataset / timed-cut setup budget this pass did not provision. The C6B-F1 .fab fix is already code + live byte-compare verified in the v0.130.0 REPORT; this pass instead prioritized the newer, higher-risk v0.131.0 F-S2/F-S3 legs (never live-proven until now).
• P0 residue: 6B's cleanup did NOT fully remove the campaign6 mount or the sonarr stackdir (both still present) — minor, non-blocking (deviates from the P0 note's "may be GONE").
• paperless tier-2 target pinned to felhom-flash (via the /stacks/paperless-ngx/backup UI control) to route around F-6C-1 for the F-S2 proof — a legitimate UI action, recorded.
• Stale target residue (felhom-flash/appdata/paperless, 16 KB, Jun-15 prior-session) cleared before 4P-5 so the migration collision guard (working correctly) didn't block the test.

---

Box state / morning recovery

• paperless-ngx torn down (real stop→remove flow + scratch-residue cleanup): containers gone, app.yaml gone, appdata/paperless + primary unit + tier-2 secondaries (flash + nas-media) removed. Pre-existing skeleton (userdata/paperless-ngx canary, empty import/paperless dirs) left as-is.
• Controller v0.132.0 healthy on demo 9201 post-teardown; other 6 apps untouched and running.
• Session cookie/CSRF wiped from the controller container /tmp.
• No code shipped this pass (findings-only). Viktor rotates the controller + hub credentials after the campaign (per the campaign rule).

---

Evidence index

• 180:~/campaign6/6C-P0.md — P0 state, auth note, full 4-paperless leg-by-leg results, F-6C-1, 5-rest observation (this doc's source ledger).
• 180:~/campaign6/ledger.md + evidence/ — 6B carry-over.
• sha256 manifests (in-run, /tmp/*.man on the guest, transient): 4P-1 source (11 files, 40587 B), 4P-2 secondary byte-compare (diff empty), 4P-5 pre/post/round-trip (all identical).
• Controller logs (demo 9201, 2026-07-14 17:3x–17:4x): tier-2 exit-23 (F-6C-1), tier-2 appdata mirror, file-restore "additive-only … dst=appdata/paperless … 1 file restored", migration complete lines.

This pass closes the v0.131.0 REPORT's "NOT live-validated (deferred to Viktor's supervised session)" list — F-S2 tier-2 backup + marker restore + storage-page size, and F-S3 scope="app" migration — all now PASS with byte-identical evidence. See §Checklist 4P-1..4P-5 and the manifests above.