# CAMPAIGN-6C — supervised planes + deep tiers + v0.130–0.132 fix-verifies (2026-07-14)

**Seed:** `6C-20260714` · **Ledger/evidence:** `180:~/campaign6/` (6C-P0.md + 6B ledger.md/evidence)
**Continuation of:** CAMPAIGN-6B. **Class:** acceptance + findings.
**Boxes:** controller **v0.132.0** both guests · agent **0.88.0** · hub **0.54.0**.

This pass ran **unattended** (operator-approved) via the controller's own API — authenticated with the
Campaign-4 credential (out-of-band, `180:~/.config/credentials`), driving the **exact endpoints the
dashboard invokes** (`docker exec felhom-controller curl 127.0.0.1:8080` + session cookie + CSRF).
This is the live-test-real-flow acceptable proxy where a rendered browser wasn't used. No credential,
R, or blob is written here. Cookie/CSRF were transient (container `/tmp`, wiped at teardown).

---

## Checklist

| Bucket | Leg | Result |
|--------|-----|--------|
| P0 | baselines (ctrl 0.132.0 / agent 0.88.0 / hub 0.54.0), `backup block rejected`=0, storage headroom | **PASS** |
| P0 | API access mechanism (session-cookie auth, no loopback bypass) | **CHARACTERIZED** |
| **4P-1** | deploy paperless-ngx (felhom-usb), ingest 3 varied docs via consume flow | **PASS** |
| **4P-2** | F-S2 tier-2: appdata leg mirrors `appdata/paperless` byte-identical | **PASS** (+ **F-6C-1**) |
| **4P-3** | F-S2 marker restore: resolved live dir, additive-only, byte-identical | **PASS** |
| **4P-4** | storage page shows paperless-ngx non-empty size (39.6 KB) | **PASS** |
| **4P-5** | F-S3 scope="app" migration usb→flash→usb, full manifest, no ghost dir, no WARN | **PASS** |
| 5-rest | dead-app alert | **INCONCLUSIVE** (wrong trigger — clean stop ≠ crash) |
| Accept #1 | C6B-F1 `.fab` ≥1 GiB full-circle byte-compare | **DEFERRED** (needs dedicated setup) |
| PHASE 3 | escrow wizard / session-CSRF UX / hub 8-tab ring (browser) | **NOT-RUN** (needs supervised browser) |
| PHASE 4-deep | F7 live cut, restic self-heal, offsite restore-verify, snapshot coherence | **NOT-RUN** (deep-setup budget) |

---

## Verdict

**GREEN for the v0.131.0 acceptance.** All five PHASE 4-paperless legs PASS — this **fully closes the
"NOT live-validated" list in the v0.131.0 REPORT** (F-S2 tier-2 backup, F-S2 marker restore, F-S2
storage-page size, F-S3 scope="app" migration round-trip), on the live demo through the real dashboard
endpoints, with byte-identical manifests at every step and **no `appdata/paperless-ngx` ghost dir and
no `declared appdata dir absent` WARN**. One new MEDIUM finding (**F-6C-1**, tier-2 to an NFS target).
The browser planes (PHASE 3) and deep tiers (PHASE 4-deep) + the `.fab` ≥1 GiB circle remain for a
supervised/deep-budget pass (6D) — the credential-gated automation ran, but those legs need a rendered
browser (operator-typed password + on-screen typed-back) or large-dataset/timed-cut setup this
unattended pass did not provision.

---

## Findings (ranked)

### F-6C-1 (MEDIUM) — tier-2 auto-target = NFS share fails at the recovery-unit leg (`rsync -a` chown)
`RunAllTier2` auto-picked **nas-media (NFS)** as paperless-ngx's off-drive target. `rsyncMirror`
(`internal/backup/tier2.go`) uses `rsync -a`, whose `-o -g` preserve owner/group → `chown` on the NFS
export fails ("Operation not permitted", root_squash) → **exit 23**, the recovery-unit leg fails first,
so `RunTier2` records failure and the **appdata leg never runs**. Same class hit calibre-web
(`teszt_enroll` mkdir permission-denied — a separate bad-mount residue).
- **Repro:** any box whose first schedulable off-disk tier-2 target is an NFS share.
- **Impact:** tier-2 silently produces no off-drive copy for the affected app (status = failure, so
  it's honest, not silent — but no protection). Real drives (felhom-usb/flash) are unaffected.
- **Not an F-S2 regression** — F-S2 (the appdata dir-name resolution) is proven correct once the
  target is a real drive (4P-2 byte-compare).
- **Suggested fix (→6D / a controller task):** for a network-fs target, drop `-o/-g` (add
  `--no-owner --no-group`, or `--chown`), or exclude NFS mounts from the auto tier-2 target pick.
- **Evidence:** controller log 2026-07-14 17:35:31 (exit 23 chown lines); `180:~/campaign6/6C-P0.md`.

### Observation (LOW, → 6D) — aggregate state "running" while main container Exited(0)
A clean `docker stop paperless-webserver` (exit 0) did NOT fire a dead-app alert in 2.5 min and the
controller reported `state=running` / health-probes "4 ok". This is **likely by design** (a clean stop
is not a crash; `downstate_test` asserts certain states must not manufacture an alert), so 5-rest is
**inconclusive with this trigger**. Worth a 6D note: the aggregate state reads "running" while the
service container is down (other containers up). A proper crash trigger (`docker kill -9` / OOM,
exit≠0) is the correct test — deferred (restart-policy race needs care).

---

## Timings (paperless acceptance, demo)

| Step | Wall time |
|------|-----------|
| deploy → 3 containers healthy | ~95 s (Postgres start_period) |
| 3-doc consume ingest (OCR + PDF/A) | ~90 s |
| tier-2 run (all HDD apps) | ~30 s |
| tier-2 file-restore (stop→copy→start→health) | 39 s |
| migration usb→flash (scope=app) | 45 s |
| migration flash→usb (round-trip) | 45 s |

---

## Deviations from the prompt

- **PHASE 3 / 4-deep / Accept #1 not run.** PHASE 3 needs a rendered browser with an operator present
  (browser-select prompt, password typed at the login screen, and the wizard's "type the two
  highlighted words read from screen") — genuinely interactive, not unattended-automatable. 4-deep
  (F7 live 6 s cut, restic mid-run kill, offsite restore-to-verify) and Accept #1 (≥1 GiB userdata app
  + encrypted `.fab` round-trip) need large-dataset / timed-cut setup budget this pass did not
  provision. The C6B-F1 `.fab` fix is already code + live byte-compare verified in the v0.130.0 REPORT;
  this pass instead prioritized the **newer, higher-risk v0.131.0 F-S2/F-S3** legs (never live-proven
  until now).
- **P0 residue:** 6B's cleanup did NOT fully remove the campaign6 mount or the sonarr stackdir (both
  still present) — minor, non-blocking (deviates from the P0 note's "may be GONE").
- **paperless tier-2 target pinned** to felhom-flash (via the `/stacks/paperless-ngx/backup` UI
  control) to route around F-6C-1 for the F-S2 proof — a legitimate UI action, recorded.
- **Stale target residue** (`felhom-flash/appdata/paperless`, 16 KB, Jun-15 prior-session) cleared
  before 4P-5 so the migration collision guard (working correctly) didn't block the test.

---

## Box state / morning recovery

- **paperless-ngx torn down** (real stop→remove flow + scratch-residue cleanup): containers gone,
  `app.yaml` gone, `appdata/paperless` + primary unit + tier-2 secondaries (flash + nas-media) removed.
  Pre-existing skeleton (`userdata/paperless-ngx` canary, empty `import/paperless` dirs) left as-is.
- Controller **v0.132.0 healthy** on demo 9201 post-teardown; other 6 apps untouched and running.
- Session cookie/CSRF wiped from the controller container `/tmp`.
- **No code shipped this pass** (findings-only). **Viktor rotates the controller + hub credentials
  after the campaign** (per the campaign rule).

---

## Evidence index

- `180:~/campaign6/6C-P0.md` — P0 state, auth note, full 4-paperless leg-by-leg results, F-6C-1,
  5-rest observation (this doc's source ledger).
- `180:~/campaign6/ledger.md` + `evidence/` — 6B carry-over.
- sha256 manifests (in-run, `/tmp/*.man` on the guest, transient): 4P-1 source (11 files, 40587 B),
  4P-2 secondary byte-compare (diff empty), 4P-5 pre/post/round-trip (all identical).
- Controller logs (demo 9201, 2026-07-14 17:3x–17:4x): tier-2 exit-23 (F-6C-1), tier-2 appdata mirror,
  file-restore "additive-only … dst=appdata/paperless … 1 file restored", migration complete lines.

**This pass closes the v0.131.0 REPORT's "NOT live-validated (deferred to Viktor's supervised
session)" list** — F-S2 tier-2 backup + marker restore + storage-page size, and F-S3 scope="app"
migration — all now PASS with byte-identical evidence. See §Checklist 4P-1..4P-5 and the manifests
above.