docs: R-165 merge built and proven at the bake; R-163 + R-175 closed, R-178 filed

07-backup-architecture.md gains §7.5.1 (S-1: the contract changed in the same
session): the ceiling §7.5 describes no longer exists for a box built from
golden >= 0.192.0, the bulkhead's replacement is recorded, and R-175 is FIXED
here rather than left standing — the bound is restated as a function of mp1
and scoped to split-layout boxes, naming all three real shapes.

Capability map: new row as IMPLEMENTED, deliberately NOT proven-live, with
the missing leg named — no box has been reinstalled from the golden, and
"the golden baked" is not "a box built from it works".

R-163 CLOSED: the ceiling it recorded stops existing. R-176(a) answered by
P1; (b) WITHDRAWN, since every node is reinstalled rather than migrated.
R-178 filed for the reinstalls, which were not done this session.

CONTEXT S-13 (the variant chosen on measurement; pruning rejected with its
reason) and S-14 (prove first, then vouch — the golden is published but
deliberately unvouched, because vouching is what makes a fresh install pick
up a layout no box has been proven from).

STATUS: plain-language section; both operator questions now answered, so the
waiting-on-you item is cleared. Two older entries trimmed so the page did
not grow.

documentation/backlog/OPEN-ITEMS.md

@@ -90,8 +90,9 @@ State: `BLOCKED` · `READY` · `WAITING-ON-OPERATOR` · `WATCHING`. Every row ha
 | **R-171** | **The boot sweep started apps whose data drive was ABSENT — a regression introduced by v0.189.0, now FIXED.** Replacing `isBootOrphan`'s container-count term with recorded intent made a drive-gate-stopped app (`compose down` ⇒ zero containers, and the gate never touches `desired_state` because it is not the customer) read as a boot orphan | **CLOSED — SHIPPED + PROVEN-LIVE** (controller v0.190.0, 2026-08-02) | — | **Reasoned from the diff, then CONFIRMED on hardware before any fix was written** (`audits/DIAG-bootrecon-drive-absent-2026-08-02.md`). The sweep found and started calibre-web with its drive unmounted, burned both attempts and handed it to the dead-app alarm — **a false alarm about an app the drive gate is deliberately holding**. The *write* hazard did NOT materialise: compose failed `mkdir …/userdata: permission denied` because the unbound mountpoint is host-root-owned and the guest is unprivileged — **an accidental protection no code owns, no test pins, and one `chown` or one privileged guest away from gone**. Fix: new consumer-side seam `bootrecon.StartGate`, **fail-safe (cannot determine ⇒ do not start)**, wired in `main.go`; `Manager.DriveLive` reuses the userdata belt's own `isMountPoint` seam so the two cannot drift. **The rule is not new** — the API's `startGatedByMissingDrive` already refused this to the customer; the sweep bypassed it by calling `Manager.StartStack` directly. Widening the window (R-157 A) made two more holders reachable, so the same seam also refuses an app held by a **quiesce** or an **in-flight app-data operation** (§8.2), reusing `quiesce.SuppressedStacks()` and a new read-only `AppStopGuard.HeldStacks()`. Held apps report as `HeldByDrive`, never `StillDown` — that is the alarm's bucket. **ID established free:** `grep -ro "R-171\b" documentation/ *.md` → 0 hits before minting | — |
 | **R-172** | ~~**A false `host_stale` alarm fires when the hub's SQLite refuses two consecutive host reports.**~~ | **CLOSED — SHIPPED + PROVEN-LIVE** (hub v0.88.0, 2026-08-02) | — | **ROOT CAUSE WAS NOT TUNING — THE PRAGMAS WERE NEVER APPLIED.** `store.New` used `?_journal_mode=WAL&_busy_timeout=5000`, which is **mattn/go-sqlite3** syntax; the driver is **modernc.org/sqlite**, whose `applyQueryParams` reads only `_pragma`/`_time_format`/`_time_integer_format`/`_txlock`/`_inttotime` and **ignores the rest without an error**. The hub ran in rollback-journal mode with `busy_timeout=0` for its entire life while its own source said WAL — a configuration asserting an invariant the code did not provide. Proof: a 128 MB open `/data/hub.db` with **no `-wal`/`-shm` beside it**. **Fix:** `?_pragma=journal_mode(WAL)&_pragma=busy_timeout(5000)&_txlock=immediate`. **`_txlock=immediate` is not optional** — `database/sql`'s `Begin()` is DEFERRED, so a read-then-write tx must upgrade its lock and a failed upgrade is `SQLITE_BUSY_SNAPSHOT`, which **`busy_timeout` does not retry**; this store has 10+ `db.Begin()` sites, all write paths. **Retry options (b) and (c) were deliberately NOT taken** — with readers no longer blocking writers a surviving `SQLITE_BUSY` would be a real signal, and a retry would hide it; revisit only on evidence. **Live:** `-wal`+`-shm` now present, **zero `SQLITE_BUSY` since rollout**, host back to `ok`, and `PRAGMA integrity_check` = `ok` with `journal_mode=wal` after three unrelated OOM restarts. **Operational consequence handled:** a WAL DB cannot be copied by taking `hub.db` alone — the break-glass retrieval in `operations/nodes.md` did exactly that and is now WAL-aware (the live `-wal` was 729 KB, i.e. a bare `cat` would have silently omitted it) | — |
 | **R-174** | ~~**The app-stop guard's crash recovery started apps onto MISSING drives — a regression in v0.189.0 code.**~~ | **CLOSED — SHIPPED + PROVEN-LIVE** (controller v0.191.0, 2026-08-02) | — | **Found by REVIEW on 2026-08-02, in code shipped 2026-08-01, and closed the same session — R-171 one path over.** `appStopGuard.SetStarter(stackMgr)` handed `Recover` the RAW stack manager, whose `StartStack` has no drive gate, and `Recover` runs **at startup** — exactly when an external drive may not have come back. So: a backup stops an app, the box loses power, the drive does not remount, and the app is started on a missing drive. The rule was not new — the API's own `startGatedByMissingDrive` already refused this to the customer; the guard bypassed it. **`bootDriveGate` could NOT be reused whole**, and the reason is recorded in the code: its holder #2 reads `bootAppStopGuard.HeldStacks()`, which during `Recover` is **the guard's own marker** — it would refuse every recovery it was meant to perform — and holders #1/#2 read package-level vars assigned AFTER `Recover()` runs, so a whole-gate reuse would be correct only by accident of nil-safety. Holder #3 is extracted into a shared `driveStartGate` with **two callers, one implementation**, and `TestBootDriveGateAndAppStopShareTheDrivePredicate` pins the delegation. **A REFUSAL IS NOT A FAILURE:** new `ErrStartRefused` + a `Refused` bucket — both keep the marker, only `Failed` alarms, because routing a deliberate hold into `NotifyBackupFailed` (customer-enabled by default) is the very R-171 false alarm this fixes. `main.go` guards on `Alarming()`, not `!= nil`, and the pre-existing seam test was TIGHTENED to require it. **Live on 9201, both directions:** drive held unmounted → `refusing to restart "calibre-web" … drive /mnt/felhom-drives/hdd_1 is not a live mountpoint`, marker retained byte-identical, zero containers started, `not alarming`; drive returned → `restarted calibre-web`, marker CLEARED. **ID established free:** `grep -ro "R-174\b" documentation/ *.md` → 0 hits | — |
-| **R-175** | **`07-backup-architecture.md` §7.5 states ONE box's size bound as if it were the fleet's.** Its headline — *"an app can be restored by Lane 1 only while its recovery unit still fits the retained space — ≈ 19 GB of app data for a file-only app, ≈ 10 GB for a DB-backed one"* — is derived from `mp1 = 20 G`. **That is demo-hp exactly, and it is not demo-felhom**, which ships `mp0 200G / mp1 50G`, where the real bound is ≈ 49 GB / ≈ 24 GB. The golden's own defaults are a third pair (`GOLDEN_DOCKER_GB=16` / `GOLDEN_SYSDATA_GB=8`), grown at provision time | **READY (S) — NEW 2026-08-02** | — | **Measured, not inferred** (`audits/SPIKE-r165-mp1-merge-2026-08-02.md` M1: `pct config 9201` on both hosts). Independent of the merge — the sentence is wrong today and will be wrong differently after R-165. **The fix is to state the bound as a FUNCTION of `mp1`, not a constant**, and to say which box any quoted figure came from. Same class as the comment-asserting-an-invariant rule: a doc stating a fleet-wide number that only one machine satisfies reads as settled and is not. **ID established free:** `grep -ro "R-175\b" documentation/ *.md` → 0 hits | CC |
-| **R-176** | **Two prerequisites for the R-165 merge are UNMEASURED, and both are cheap.** (a) Whether a **pre-merge archive** (carrying `mp1`) restore-tests cleanly into a **merged-layout** guest — reading `mountParity` (`felhom-agent/internal/reconcile/restoretest.go:347`) says it should, because the restore recreates `mp1` from the archive so archive and restored guest agree; **that was reasoned from source and never executed.** (b) The in-place per-box migration (move `<mp1>/felhom-data` onto `mp0`, drop the slot, verify) has **never been rehearsed even once**, so "is the box restorable at every point of it?" is currently unknown | **READY (S) — NEW 2026-08-02** | blocks R-165 landing safely | **Filed because this project's own record is that FOUR production designs specced against unvalidated mechanisms were all wrong** — which is exactly why R-165's own spike refused to design. Both are one command on a **Tier-0** box (D-d: both demo boxes are disposable). (b) is only required work if Peti's box turns out to need migrating rather than reinstalling — the hub cannot answer that (M5: `peti-felhom` exists as a customer with **no host in the register**), so it is the operator's input. **ID established free:** `grep -ro "R-176\b" documentation/ *.md` → 0 hits | CC |
+| **R-175** | ~~**`07-backup-architecture.md` §7.5 states ONE box's size bound as if it were the fleet's.**~~ | **CLOSED — FIXED 2026-08-03** (same pass as R-165) | — | **Measured, not inferred** (`audits/SPIKE-r165-mp1-merge-2026-08-02.md` M1: `pct config 9201` on both hosts). Independent of the merge — the sentence is wrong today and will be wrong differently after R-165. **The fix is to state the bound as a FUNCTION of `mp1`, not a constant**, and to say which box any quoted figure came from. Same class as the comment-asserting-an-invariant rule: a doc stating a fleet-wide number that only one machine satisfies reads as settled and is not. **ID established free:** `grep -ro "R-175\b" documentation/ *.md` → 0 hits **FIXED.** §7.5 gained a **7.5.1** which (a) states plainly that the bound is a FUNCTION of `mp1` and applies only to a box still on the split layout, naming all three real shapes, and (b) records that the ceiling itself has been removed by R-165 for boxes built from golden ≥ 0.192.0. Fixed in the same pass as the merge rather than filed and forgotten, because the section would otherwise have been wrong in two ways at once | CC |
+| **R-176** | **Two prerequisites for the R-165 merge are UNMEASURED, and both are cheap.** (a) Whether a **pre-merge archive** (carrying `mp1`) restore-tests cleanly into a **merged-layout** guest — reading `mountParity` (`felhom-agent/internal/reconcile/restoretest.go:347`) says it should, because the restore recreates `mp1` from the archive so archive and restored guest agree; **that was reasoned from source and never executed.** (b) The in-place per-box migration (move `<mp1>/felhom-data` onto `mp0`, drop the slot, verify) has **never been rehearsed even once**, so "is the box restorable at every point of it?" is currently unknown | **(a) ANSWERED 2026-08-03 (P1: PASS). (b) NOT REQUIRED — operator ruling: every node is reinstalled, none migrated** | blocks R-165 landing safely | **Filed because this project's own record is that FOUR production designs specced against unvalidated mechanisms were all wrong** — which is exactly why R-165's own spike refused to design. Both are one command on a **Tier-0** box (D-d: both demo boxes are disposable). (b) is only required work if Peti's box turns out to need migrating rather than reinstalling — the hub cannot answer that (M5: `peti-felhom` exists as a customer with **no host in the register**), so it is the operator's input. **ID established free:** `grep -ro "R-176\b" documentation/ *.md` → 0 hits **UPDATE 2026-08-03.** **(a) is measured and passed** — `audits/SPIKE-r165-phase0-2026-08-03.md` P1: a real pre-merge archive (`mp0+mp1`, confirmed from its own vzdump log) restore-tested on demo-hp, `pass: true`, `mount_parity: ok`, 84 s, with `mountParity` untouched. One limit stated rather than glossed: it ran with the pre-merge agent because the merged one did not exist yet, and the comparison is archive-vs-its-own-restore which never consults the host layout — **re-run it once against agent v0.120.0**, which is one command. **(b) is withdrawn, not deferred:** the operator ruled that every node is REINSTALLED rather than migrated in place (both demo boxes are Tier 0; the colleague's box carries none of our customer data and is clean-installed in a few weeks), so the in-place migration rehearsal has no consumer. Recorded explicitly rather than silently skipped | CC |
+| **R-178** | **The merged golden (0.192.0) is built and published but NO BOX HAS BEEN REINSTALLED FROM IT, and it is deliberately UNVOUCHED.** `build-golden.sh` v3.0.0 baked it with variant V-c and every retargeted assertion passed on the real bake (`including mount point mp0 ('/var/lib/felhom')`, no `mp1` line, `both paths are ONE filesystem`); it is in the registry (HTTP 200, sha `54e2a4c431daf580…`). What has NOT happened is Part 4: reinstall each demo box from it and prove claim → deploy an app → back up → restore | **READY (M) — NEW 2026-08-03** | blocks R-165 reaching PROVEN-LIVE; blocks the capability-map row | **The golden is UNVOUCHED ON PURPOSE and that is the safe state**, not an oversight: vouching is what makes a fresh install pick it up, so vouching a golden no box has been proven from would put an unproven disk layout in front of the next install anywhere. **Prove first, then vouch** — the bake script's own output treats the hub record as a separate deliberate step for this reason. **Everything else for the merge is shipped and green:** controller v0.192.0 (the B2 floor) is live on 9201, agent v0.120.0 is live on BOTH hosts, and `felhom-host-install.sh` computes the single grow from the thin pool. So a reinstall is now a self-contained piece of work with no code left to write. **Order matters: ONE box at a time**, demo-hp first, proven end to end, and only then demo-felhom — two in parallel leaves no working reference to compare against. Note demo-felhom carries the PBS-DR/offsite tier, so it is the one whose backup chain a reinstall actually disturbs. **ID established free:** `grep -ro "R-178\b" documentation/ *.md` → 0 hits | CC |
 | **R-177** | **There is no operator-triggerable "run the fill check now" path.** `fill-watch` is reachable only on its daily 03:30 schedule plus the once-at-startup run added in controller v0.191.1 — so the only way to exercise it on demand is to restart the controller | **READY (S) — NEW 2026-08-02** | — | **Noticed while live-validating R-167 on 9201, not by a failure.** It cost a controller restart per observation during validation, and it costs the same on a support call: after a customer frees space, nobody can confirm the warning has cleared without restarting their controller or waiting until 03:30. **Partially mitigated already** — v0.191.2 makes every run log a positive observable (`checked N filesystem(s), M unreadable/skipped, K notification(s)`), so at least a run that DID happen is visible; the gap is triggering one. The scheduler has `GetJobs` but no run-now, so this is a general affordance, not a fill-watch one — **scope it as "run a named scheduler job now", operator-gated.** **ID established free:** `grep -ro "R-177\b" documentation/ *.md` → 0 hits | CC |
 | **R-173** | **The hub's SQLite PVC is excluded from every Longhorn backup job.** `pvc/hub-data` carries `recurring-job-group.longhorn.io/default: disabled`, and `backup-daily` + `backup-weekly` (04:00 / Sun 05:00) are the ONLY recurring jobs and both target the `default` group — so the 128 MB `/data/hub.db` has **no volume-level backup**. That database holds `host_recovery` (every managed box's break-glass root password), `host_escrow` + `host_escrow_superseded` (escrow custody), `host_pbs_secrets`, `customer_configs`, `dr_recipe` and the wg endpoints/peers — i.e. the material several documented recovery routes depend on | **READY (M) — NEW 2026-08-02** | — | **Noticed while checking the blast radius of the R-172 WAL change, not by a failure** — the WAL work needed to know who copies this file, and the answer turned out to be nobody on a schedule. **Establish before designing:** (a) whether the exclusion is deliberate (a 1 Gi RWO Longhorn volume snapshotting a 128 MB SQLite file is cheap, so the label looks like a leftover rather than a decision) and by whom; (b) whether anything else backs it up out-of-band that this census missed — the `_recovery-inventory-2026-07-28.md` records a MANUAL hot copy, which is not a backup. **When it is designed, it must be WAL-aware** (R-172): a volume snapshot of a live WAL database is crash-consistent and replays on open, which is fine, but any file-level copy must take `hub.db-wal` too or it silently loses the newest writes. **Grep establishing the ID was free:** `grep -ro "R-173\b" documentation/ *.md` → 0 hits | CC |
 | **R-158** | ~~**A local Tier-1 app-data backup failure reaches no hub channel.**~~ | **CLOSED BY R-167 — SHIPPED + PROVEN-LIVE** (controller v0.191.0 + hub v0.89.0, 2026-08-02) | — | **Closed by the wire it named; no second row was filed for it** (R-167 subsumes and widens it). New `unitNotify` seam + `SetUnitNotify` beside the manager's existing three, called from `captureAllRecoveryUnits` **per app with the loop continuing**, carrying the target filesystem's used/free bytes at the moment of failure — the cause is usually a full filesystem and those numbers answer *why* without an operator logging in. **ROUTED TO THE OPERATOR, NOT `backup_failed`, AND THAT OVERRIDES THIS ROW'S OWN PROPOSAL.** The proposal above said *"emitting the existing `backup_failed`"*; that type carries a `customerMessages` entry AND sits in `settings.DefaultEnabledEvents`, so it would email the customer in Hungarian about a failure they cannot act on — precisely the mistake R-97a avoided by minting `whole_guest_backup_failed`. **Decision D-c routes it to the operator and D-c wins.** New `recovery_unit_capture_failed` in `allowedEventTypes` **and** `notify.operatorOnlyEvents`; `notify.IsOperatorOnly` added so ONE test pins both registers (allowlisted-but-not-operator-only is invisible when they are checked separately — the v0.78.0 defect). **Red-proof:** removing the register entry shows the customer being emailed. **Live on 9201:** two events accepted and stored, `operator | sent`, and the positive observable `customer | recovery_unit_capture_failed | skipped | operator_only` read from the hub's `notification_log` | — |
@@ -99,9 +100,9 @@ State: `BLOCKED` · `READY` · `WAITING-ON-OPERATOR` · `WATCHING`. Every row ha
 | **R-160** | **gramps-web persisted three paths and wrote to none of them.** `/app/data` appears nowhere in the image's environment; the accounts DB (`GRAMPSWEB_USER_DB_URI`) and **the family tree** (`GRAMPS_DATABASE_PATH=/root/.gramps/grampsdb`) both landed in the writable layer. Upstream persists **eight** paths; the template persisted three, one a phantom. | **SHIPPED** (`templates/gramps-web/docker-compose.yml`, 2026-08-02) | — | **Severity above papra's, and worth keeping visible:** papra loses documents the customer may hold elsewhere; gramps-web loses **the family tree — the artefact built inside the app, of which no other copy exists by construction.** Evidence: `app-catalog-felhom.eu/audits/persistence-sweep-2026-08-02/` | CC |
 | **R-161** | **The volume-persistence gate is enforced by CONVENTION, not automatically.** The catalog repo has no CI of any kind (`.gitea/workflows`, `.github`, drone/woodpecker — searched, none exists). | **REDUCED SCOPE — open** (operator ruling 2026-08-02) | a second person touching templates | **RULED. Both obvious enforcement points were rejected for measured reasons.** *Controller-side at template load:* rejected because such a check can only read the file, and a static audit of all 53 templates reports the catalog clean **including papra** — **it would pass on the exact defect it exists to catch**; the property is decidable only at runtime. *CI:* rejected for now — neither repo has any, and there are no users yet. **SHIPPED instead** (`app-catalog-felhom.eu` `fd7747d`): `scripts/catalog_gates.py`, ONE entry point running all three gates, non-zero exit on any failure, **mandated in the catalog's `CLAUDE.md`** the way `site_gates.py` is. Rationale for the record: of this project's gates, the only ones that ever get run are those with a single entry point named in a CLAUDE.md — `site_gates.py` is run, R-29's three orphans are named nowhere and have stopped nothing. **What remains open is only the automatic half:** this is convention, run by a person, and that is sufficient while one person touches templates. Revisit when a second does **UPDATE 2026-08-02:** `catalog_gates.py` gained `--fast` (gate 1 only — the network and runtime gates are deliberately NOT in a hook: a push that pulls images and starts containers gets bypassed within a week and the bypass becomes the habit) and `.githooks/pre-push` now runs it. **The automatic half now has a designated successor row: R-168** (Gitea Actions runner). This row stays open at its reduced scope — the runtime gate remains a deliberate periodic run **UPDATE 2026-08-02 (second):** the automatic half now EXISTS — R-168's runner executes `catalog_gates.py --fast` on every push to this repo (measured: run #1, `image-pin gate OK — 53 templates`, with the two runtime gates announced as skipped and their own output absent from the log). This row's *original* scope — the RUNTIME volume-persistence gate — is deliberately still NOT automatic and should stay that way: CI that pulls 53 images on every push gets disabled. It remains a periodic run | operator |
 | **R-162** | **`docker diff` is the gate's only witness, and its failure mode is quiet.** The gate's power comes from `docker diff` excluding mounted paths, which makes "in the writable layer" mechanically decidable — an implementation detail of the overlay driver. On a driver where `docker diff` is unsupported or lies, the gate degrades to the mount-occupancy and writability legs **and would not say so**. | **WATCHING** — a limitation, not a defect | — | It **fails closed**: the canary self-test would stop reporting BROKEN and the gate would then refuse to report at all. What is wrong is the message — it would blame the prober rather than the driver. Revisit only if a non-overlay storage driver ever ships | CC |
-| **R-163** | **`mp1` is RETENTION, not staging — and it is sized as if it were neither.** A recovery unit is the KEPT copy on the app's **own** drive (`GetAppDrivePath`, `internal/backup/backup.go:245-255`); for an app with no `HDD_PATH` the namespace falls back to the system SSD — *"the SSD-only system-data fallback"* (`internal/appbackup/paths.go:26-27`). There is **no post-copy deletion**: the only prune is F5 (`backup.go:1053-1112`), residue on OLD drives when an app MOVES. So `mp1` (**20 G**) retains the units of every driveless app, while `mp0` permits **50 G** of volumes — and a DB app's unit is up to **~2×** its data (volume tar **plus** SQL dump; measured 21.1 GB → 40.2 GB). `--sysdata-grow` defaults to **0** (`felhom-agent/cmd/felhom-agent/main.go:178`) and is **not** derived from the physical drive; demo-hp's real guest 9201 ships `mp0 50G / mp1 20G`. | **RE-FRAMED 2026-08-02 — open, no longer waiting on a ratio** | — (the sizing question is answered; the work is **R-165**) | **RE-FRAMED, NOT CLOSED (operator decision D-a, 2026-08-02 — `CONTEXT.md` S-5).** The row asked *what ratio should `mp1` be?* and that question is **withdrawn rather than answered**: `mp1` is merged into `mp0` so local recovery units share the app-data area and the ceiling stops existing — a bigger number is the same wall further away. **This row stays open as the record of the constraint** (what `mp1` is for, what it gates, and the measured 2× DB-app unit size) **until the merge lands**, because until then every consequence below is still live on every box. **The work is R-165; the warning that must ship with it is R-167.** Original finding, unchanged, follows. **No number is proposed here deliberately.** What is recorded is the constraint and its blast radius: **`mp1` gates the whole app-data chain**, because Tier-2 mirrors the unit *"(always)"* from `RecoveryUnitPath` (`internal/backup/tier2.go:302,368`) and Tier-3 carries it too — a unit that cannot be written has nothing for either to copy. Bounded on the other side: a unit holds **volume tars + DB dumps only, never `mp8` userdata** (`internal/backup/recovery_unit.go:20-25`), so a 1 TB photo library is never in one. **This bounds D5's Lane-1 independence** — see `architecture/07-backup-architecture.md` §7.5. Overflow itself is SAFE (R-158's measurement: refuses per app, last good unit preserved byte-identical) — what is missing is the warning, which is R-158 (widened to R-167) | CC |
+| **R-163** | ~~**`mp1` is RETENTION, not staging — and it is sized as if it were neither.**~~ | **CLOSED by R-165 — the ceiling it describes no longer exists** (golden v3.0.0, 2026-08-03) | — (the sizing question is answered; the work is **R-165**) | **Closed, not merely re-framed.** This row was the record of a constraint that was to stay open *"until the merge lands"*. It has landed: the golden ships ONE data volume, so there is no separate 20 G area for a driveless app's recovery unit to outgrow, and the free space an app can use is the box's actual free space. **What replaced the constraint is recorded on R-165**: the bulkhead the partition also provided is now B2's explicit capture floor (controller v0.192.0), and the measured 2× DB-app unit size this row documented is what justifies the floor's reserve being a reserve rather than a working budget. **Caveat carried forward, deliberately:** no box has been reinstalled from the merged golden yet (R-178), so every box in the field still has the split layout and this row's consequences remain live ON THOSE BOXES until they are reinstalled. Original finding unchanged below | CC |
 | **R-164** | **C2's chain: the DB volume tar cannot be dropped until a SOUND dump predicate exists.** The unit carries both a volume tar and a SQL dump; the restore uses **both** — the dump is authoritative and replayed *after* the tar so it WINS (F17), with only the DB service up (R-47) — `internal/backup/restore_unit.go:262-266`. Dropping the DB container's tar would halve DB-app units **and** close the R-127(b) initdb-skip password trap (restored PGDATA ⇒ `POSTGRES_PASSWORD` ignored). | **BLOCKED** — on the predicate | a dump-validity predicate that is not `accounts has rows` | **The obvious gate is DEAD, measured:** `ValidateDump` warns when the `accounts` table is empty, and that warning was **correct** — the live DB genuinely had 0 accounts, and seeding one stopped the warning and put the row in the dump. But **a fresh appliance legitimately has zero accounts**, so promoting that predicate to a gate would **block every new customer's first backup**. Order: (1) a sound predicate — dump vs **live** per-table counts, not an absolute expectation; (2) warn→gate; (3) tar-drop. **Until (1), the tar is load-bearing** — not because dumps are bad, but because nothing can yet prove one is good. Pairs with **R-127** | CC |
-| **R-165** | **Merge `mp1` into `mp0` — the dedicated 20 G backup partition stops existing.** Operator decision **D-a**, 2026-08-02 (`CONTEXT.md` S-5). Local recovery units share the app-data area instead of holding their own fixed ceiling, so the wall R-163 describes is removed rather than moved further away. Guest 9201 on demo-hp ships `mp0 50G / mp1 20G` today | **READY (M) — NEW 2026-08-02** | — | **Two conditions travel WITH the decision and are not optional.** **(1) Before any external install.** It changes the **disk layout**, so it is a fresh-install shape while there are no external boxes and a per-box migration after — and the decision's cheapness is entirely a function of that ordering. **(2) It removes a wall that currently fails safely**, so **R-167** (D-c: fill warning + failure alert) lands in the same step, never after: today an app that outgrows `mp1` is refused per app with the last good unit preserved byte-identical (R-158's measurement), and after the merge the same overflow consumes the space the app itself is using. Touches the installer/agent guest shape (`--sysdata-grow` defaults to **0** and is not derived from the physical drive, `felhom-agent/cmd/felhom-agent/main.go:178`) and the golden. **Does NOT close R-163** — that row is the record of the constraint and stays open until this lands **MEASURED 2026-08-02 — `audits/SPIKE-r165-mp1-merge-2026-08-02.md`; no layout was touched.** **M1: "the layout" is not one thing** — demo-felhom ships `mp0 200G / mp1 50G`, demo-hp `50G / 20G`, the golden `16G / 8G`; any plan expressed as a fixed pair is already wrong for one of the two boxes (this also invalidates §7.5's fleet-wide bound → **R-175**). **M2: the majority of `mp1` is NOT Tier-1** — on demo-felhom 1.7 G of 2.0 G is Tier-2 mirrors, plus the `userdata/import` drop zone which is on the system drive by CONTRACT (R-75); a plan accounting only for the units is wrong. Observed fill is 5% / 1% — the constraint is a ceiling problem, not a current-fill one. **M3: three assertions break, and the golden fails closed on the split in FOUR places** (`build-golden.sh:126,130` separate-mount + `:315,319` vzdump-exclusion guards), not one; the archive scope `rootfs+mp0+mp1` stays complete after the merge; `mountParity` holds for new archives. **M4 — THE BULKHEAD IS THE REAL COST:** today an overflow is refused per app with the last good unit byte-identical AND CANNOT REACH `/var/lib/docker`; after the merge it can, and a full Docker data-root is a stopped box, not a degraded one. Four candidate replacements costed (reserve / capture-path refusal / project quota / warnings-only); **not chosen — operator's ruling.** **M5: D-a's condition (1) is currently SATISFIED** — no external box is in the hub's host register (only the two ONLINE demo boxes, both **Tier 0 and reinstallable**, so migration cost for the measurable population is ZERO). **`peti-felhom` exists as a customer with NO host in the register, so its layout is UNMEASURED** and was not contacted (D-d). **Recommendation: S1 (one volume, two directories) + B2 (a refusal threshold in the capture path), as a fresh-install shape with the demo boxes REINSTALLED.** **Two prerequisites are unmeasured → R-176.** **STOPPED for the operator's ruling; the merge is a supervised session.** | CC |
+| **R-165** | ~~**Merge `mp1` into `mp0` — the dedicated 20 G backup partition stops existing.**~~ | **SHIPPED — golden `build-golden.sh` v3.0.0 + agent v0.120.0 + controller v0.192.0 (B2), 2026-08-03. NOT YET PROVEN-LIVE: no box has been reinstalled from the merged golden** | — | **Variant V-c chosen by the operator on MEASURED evidence, not by reading** (`audits/SPIKE-r165-phase0-2026-08-03.md`): one volume at the NEUTRAL path `/var/lib/felhom`, with `/var/lib/docker` and `/mnt/sys_drive` both binds of subdirectories. Three shapes were built and rebooted; **all three boot, reboot 3/3, give ONE `df` figure and keep a container's `statfs("/")` on the merged volume — the ordering worry that motivated the probe did not materialise.** They differ only in which documented guarantee they break: volume-at-`/var/lib/docker` puts customer backups INSIDE Docker's data-root (so the ordinary "clear /var/lib/docker" reflex destroys every local unit); volume-at-`/mnt/sys_drive` puts Docker's ENTIRE data-root under `/mnt`, which the controller container mounts wholesale — **measured: it then sees `/mnt/sys_drive/docker`**, falsifying the bootstrap's own scoping claim. V-c breaks neither. **P1 answered R-176(a):** a pre-merge archive (`mp0+mp1`) restore-tests clean with `mount_parity: ok` in 84 s; `mountParity` was not weakened. **P3: the four golden assertions were RETARGETED, never deleted, and each was RUN against a deliberately wrong shape — 8 checks, 8 passed**, including a NEW 2b asserting both paths are ONE filesystem (which catches the S2 shape the spike ranked worse than the split) and a new guard for a leftover `mp1` (the old "was mp1 excluded?" pattern could no longer match — a guard that cannot match has silently stopped guarding). **B2 shipped first, in controller v0.192.0**: a two-term capture floor (97% / 1 GiB) in `fillwatch`'s shape, deliberately beyond its critical band so the customer is always warned before a refusal; it refuses per app and **never deletes**, because nothing here is generational. **Golden 0.192.0 is published (registry HTTP 200, sha `54e2a4c4…`) but DELIBERATELY NOT VOUCHED** — vouching is what makes fresh installs pick it up, and the right order is prove-then-vouch. **Remaining: reinstall both demo boxes from it, prove end to end, then vouch → the work is R-178** | CC |
 | **R-166** | ~~**App state gets a desired/observed model with its own store.**~~ Operator decision **D-b**, 2026-08-02 (`CONTEXT.md` S-5) | **SHIPPED + PROVEN-LIVE** (controller v0.189.0, 2026-08-02) | — | **Both blocking facts were established at source before any code was written, and the answers changed the shape.** **(a) Does a crash-safe journal already exist for the in-flight case?** YES, twice — `internal/quiesce/quiesce.go` (marker + `Recover`, proven on live hardware by Campaign 8 fault 10) and `internal/stacks/migrate.go` (`migration.json` + `RecoverMigration`) — but **neither covers the app-data path**: `DumpAppVolumesSafe` stopped and restarted an app with **no marker, no journal and not even a `defer`**. So the pattern existed and the coverage did not; `backup.AppStopGuard` copies the proven shape into its **own** file (one file, one writer). **(b) Is the SQLite store reachable?** Irrelevant, and deliberately unused: `metrics.db` is optional by design (the controller runs with it absent), and operational state must not live in a store designed to be droppable. **Shipped:** tri-state `desired_state` in `app.yaml` written ONLY by the customer's action (API action switch, `DeployStack`, `UpdateOptionalConfig`'s redeploy branch, `.fab` import — a 14-caller census established that `StartStack`/`StopStack` must NOT be writers); `isBootOrphan` reads intent instead of `len(Containers) > 0`; **absent means UNKNOWN, never running**, so a legacy `app.yaml` keeps byte-identical pre-v0.189.0 behaviour; running-only backfill. D-b's every-container requirement was already met by `aggregateState` and was NOT re-implemented. **Live on 9201:** all three flows (stop survives a restart; a zero-container `running` app is recovered by name; a legacy app.yaml is skipped and never inferred as stopped). **Also fixed en route:** `SaveAppConfig` rebuilt `AppConfig` field-by-field (the R-100 shape) and would have dropped the new field on every save across nine call sites | — |
 | **R-167** | ~~**Storage monitoring and backup alerts.**~~ | **CLOSED — SHIPPED + PROVEN-LIVE** (controller v0.191.0/.1/.2 + hub v0.89.0, 2026-08-02) | — | **Operator decision D-c. It shipped BEFORE the R-165 merge, not with it** — D-a's condition (2) says the monitoring lands in the same step and never after, and landing it first is strictly better and costs nothing. **Customer half:** new `internal/fillwatch`, per FILESYSTEM (never per app — one full disk holding ten apps would fire ten times). **It emits the PRE-EXISTING `disk_warning`/`disk_critical` pair, which was allowlisted, copy'd, in `DefaultEnabledEvents` and checkbox'd with NO PRODUCER IN ANY REPO** — a complete customer pipeline with no producer, the **sixth** *built-but-never-wired* instance here; minting a new near-duplicate type would have left it inert forever. **Two threshold terms, whichever trips first** (85% / 5 GiB; critical 95% / 2 GiB) because a percentage alone lies at both ends of this fleet's size range — **proven live: the critical crossing fired on the FREE-BYTE term (1.7 GB) at only 91% used.** Edge-triggered on escalation, state persisted, hysteresis dead zone at 75% / 7 GiB pinned by a test; a nil usage read never warns and never clears one (§8.4). The hub's two generic `customerMessages` entries were **removed** — `FormatCustomerEmail` prefers the entry over the message, so keeping them would discard the drive label and the byte figures. **Operator half: see R-158.** **Live on 9201, all three flows:** `disk_warning` then `disk_critical` both `customer | sent` with the Hungarian rendered, exactly two events across three boots (the edge trigger held on the one between), then a silent clear that re-armed. **v0.191.1** added the once-at-startup run (Daily/Every both wait for their first tick, so a box BOOTING over the line would have stayed silent up to 24 h — the R-100 shape); **v0.191.2** added a per-run positive observable, earned when a quiet run during this session's own validation proved unreadable as evidence. Follow-ups: **R-177** (no run-now path) | — |
 | **R-168** | ~~CI: no runner exists, and with trunk-based pushes CI can DETECT but not BLOCK~~ | **SHIPPED — and the alarm is DEMONSTRATED** (2026-08-02) | — | **Runner live**: `homelab-manifests/gitea-system/act-runner.yaml`, an unprivileged host-mode `act_runner` in `gitea-system`, one owner-scoped registration serving all four repos (measured: tasks 7-10 all claimed by `felhom-gates-runner`). `.gitea/workflows/gates.yml` in each repo runs that repo's entry point with `--fast` and nothing else; no `uses:` step anywhere. **Six probes, all answered, none STOPped** — `audits/SPIKE-ci-runner-2026-08-02.md`. The two that changed the design: **P2** (stock image has git but NO python3 → custom image `felhom-act-runner:0.1.0`, base pinned, python3 and nothing else) and **P6** (a runner that loses `/data/.runner` re-registers and leaves a dead record behind → the PVC is load-bearing, measured both ways). **P5 is the one that mattered**: a failed run produced NO mail, NO notification row and NO log line from Gitea, so the run now sends its own alarm via Resend and prints the provider's accepted id. **Proven end to end, not asserted**: a deliberately broken commit pushed with `--no-verify` → run #6 `failure` → `RESEND-ACCEPTED id=5ff34766-c5f8-4588-8104-08296aeb45ab`. Posture shown from the live pod spec: `privileged: false`, all caps dropped, no docker socket, no hostPath, `automountServiceAccountToken: false`, sized at half Gitea's limits so it cannot crowd out the service holding every repository on the same node. **The standing limit stays true and is written into the manifest and every workflow: it DETECTS, it does not BLOCK** — making it block is → R-169 | — |