felhom.eu/documentation/backlog/OPEN-ITEMS.md

OPEN-ITEMS — the single source of truth for open work

Rebuilt 2026-07-27 by read-only triage. ROADMAP.md keeps the full history and reasoning; this page keeps only what is open, and it is the file to read first. Root REPORT.md is per-session and overwritten — nothing durable may live only there; a session that must not clobber it writes a non-overwritten REPORT-<topic>.md sibling instead (CLAUDE.md:82-87), of which 14 now exist.

State: BLOCKED · READY · WAITING-ON-OPERATOR · WATCHING. Every row has an owner.

ID	What	State	Blocked on	Next action	Owner
R-88a	Failing backup re-quiesces every 5 min, no backoff	SHIPPED (controller v0.176.0, 2026-07-27)	—	Live on both boxes; breaker 15m→4h, per-tier, never permanent	—
R-88b	/backup/due cannot say unknown	SHIPPED + PROVEN-LIVE (agent v0.105.0 + controller v0.178.0, 2026-07-27)	—	age_state=unknown captured on real hardware during a deliberate ep0 outage; controller deferred, zero app stacks stopped	—
E-2d	Prove E-2 on a fresh VM — a real felhom-host-install.sh 1.22.0 run, Case B naturally, a claimable customer, then add a drive (the offer) and unplug it (backup_target_absent end-to-end)	CLOSED — PARTIALLY PROVEN (2026-07-29)	—	C1, C2 proven (audits/E2D-fresh-vm-2026-07-29.md); C3, C4 proven live (audits/SESSION-C-2026-07-29.md); C5 FAILED → R-116 — the gate fires and an alarm reaches the hub, but it is the generic event, so the alarm and its recovery cannot be paired. R-116 is the single named open leg; per the Session-C runbook §9, decided in advance, a failed claim closes the item as partially proven rather than triggering a re-run. Both audits carry the full record — the local-lvm fence, the ISO/PAIRING derivation, the Phase 0 answers, the per-claim observables and the teardown evidence — and are the place to read it, not this cell. The arc's actual definition of done is R-106 + R-109, R-108 and D5, none of which this detour touched	CC
R-94	A hand-synced version constant drifts, and the gate that would catch it is never run	CLOSED — SHIPPED (hub v0.87.0, 2026-08-02)	—	All three legs closed. (a) closed by DELETION, not derivation — deriving is not achievable honestly: the Setup command fetches felhom-host-install.sh at RUN TIME from a website that git-syncs main every 30 s (R-110), so no build-time value in the hub can be true, and a number that is wrong carries a version number's authority while being a guess. The const, the pageData.ScriptVersion field, its assignment and the rendered label are gone; a NOTE stands where the const was so it is not helpfully re-added. (b) hostinstall_gates.py gate 1 INVERTED — it now asserts the hub carries no host-install version literal, in six code shapes across every .go/.html under hub/; and the gate is now invoked, by scripts/repo_gates.py and the pre-push hook (→ R-29). (c) the tautological render_test.go:219 assertion is deleted, not replaced — there is no version to assert. It was demonstrated PASSING with the const at 9.9.9 while the script was 1.22.0. The label had been wrong for 19 days (since 2026-07-14)	—
R-110	main is the installer's publish channel — there is no staging. manifests/webpage.yaml git-syncs /scripts/ from --branch=main on a 30 s period and nginx serves that working tree directly (location /scripts/, root …/current). So pushing scripts/felhom-host-install.sh is publishing it: within thirty seconds it is what every subsequent felhom-bootstrap.sh fetch (scripts/iso/felhom-bootstrap.sh:96) and every operator-run day-0 command (customer_unified.html:563) receives. There is no tag, no pinned-version path, no staging copy and no rollback other than another push — for the artifact that runs as root on a virgin box, the single most privileged thing Felhom ships	CLOSED — SHIPPED (installer v1.23.0, 2026-08-03)	—	Two consequences worth stating: E-2d is not a gate before exposure — 1.22.0 has been the live installer since it hit main on 2026-07-29 — and the precaution recorded on the old R-94 row as "do not point every new box at an installer that has never run" was never available to take. Open question for the operator, not a defect to fix blind: whether /scripts/ should serve a pinned release (tag-tracked path, or a versioned directory with the customer command naming a version) or whether main-tracking is the accepted shape for a one-operator product. Exposure today is zero — there are no boxes installing — which is exactly why it is cheap to decide now. SECOND INSTANCE, found 2026-07-29 by the E-2d run and filed here rather than as a new ID: felhom-host-install.sh fetches nine files from raw/branch/main (:2072–:2206) and the hub manifest vouches a sha for exactly one (wrapper_sha256 → felhom-pbs-apply; re-checked this run, no drift). E-2a's felhom-backup-target-apply (:2116) is installed 0755 to /usr/local/sbin and root-fenced in sudoers, validated only by bash -n — a root-executed artifact taken from main with no pinned integrity, which is this row's class exactly. OPERATOR RULING 2026-08-03 — option (b) chosen: the publish channel moves from main-tracking to a TAG. Publishing becomes moving the tag, and rollback becomes moving it back — the property main-tracking cannot have at any price. Recorded here, not built this session, by instruction. The ruling carries a condition that decides whether the fix works at all: it must cover BOTH channels. (i) the nginx-served /scripts/ git-sync (manifests/webpage.yaml, --branch=main, 30 s period) that every felhom-bootstrap.sh fetch and every operator day-0 command reads, AND (ii) the nine files felhom-host-install.sh fetches from raw/branch/main (:2072–:2206), of which the hub vouches a sha for exactly ONE. Fixing only (i) leaves a tagged installer pulling nine untagged files from main at run time — a staging story that is false in the place it matters most, since one of those nine (felhom-backup-target-apply) is installed 0755 into /usr/local/sbin and root-fenced in sudoers, validated only by bash -n. Exposure is still zero (no boxes installing), which is exactly why it stays cheap. CLOSED 2026-08-03 — installer v1.23.0, and the both-channels condition was HONOURED, but not in the shape the ruling assumed. The spec's mechanism for channel 2 rested on a factual error, found by reading the code: the run-time fetches are sixteen, not nine, and they come from the felhom-agent repo, not from felhom.eu — so no tag on this repo could ever have covered them, and §8.1's "derive the tag from SCRIPT_VERSION" was unimplementable for them. Operator ruled on the alternative: pin them to the agent version being installed, which the installer already resolves from the hub manifest and already sha-verifies. fetch_raw now fetches raw/tag/v$ART_AGENT_VER/, with no fallback to a branch — a vouched version whose tag is missing fails loudly, because a silent fallback is the appearance of control with none of it. That also fixed a latent skew → R-183. Channel 1: manifests/webpage.yaml split into TWO git-syncs — the website still tracks main at 30 s (a copy edit must never need a release), /scripts/ tracks installer-v1.23.0. Both trees are seeded by init containers, so a fresh pod is not Ready until the tag is checked out and there is no 404 window; maxUnavailable rounds to 0 on one replica, so a failed scripts-init leaves the OLD pod serving — the failure direction is no update, never no /scripts/. Channel 3 needed no change, and that is recorded rather than left as a silence: https://felhom.eu/scripts/felhom-host-install.sh never carried a ref — the ref lives in the manifest — so felhom-bootstrap.sh and the hub's day-0 command follow the tag with no edit and no hub version bump, which is why §1's no-bump rule was never in tension. PROVEN LIVE, both scenarios, by HTTP against the real URL. P-A (measured before the manifest was touched): git-sync v4.4.0 follows a tag and notices a MOVED one — update required … local:fb65202 remote:8360f94 → updated successfully, one period. Scenario A: a real push to main without moving the tag — the website tree advanced to the new commit while the scripts tree stayed put, the served sha stayed byte-identical (2f859555…) and a marker comment deliberately planted in that commit was absent from the served URL. Scenario B: moving the tag published it in ~40 s (sha → ea2b4aa9…, marker present), and moving it back rolled it back to exactly the pre-publish sha with the marker gone; https://felhom.eu/ returned 200 throughout. Gate 6 in hostinstall_gates.py pins all of it structurally with no network, so it stays in --fast and runs in CI: no raw/branch/ ref in the installer, fetch_raw still pinned, the manifest still splitting tag-vs-main. It deliberately does NOT assert that a tag exists for the current SCRIPT_VERSION — that would go red on the very push that bumps the version, before publishing, and publishing being a separate act is the whole ruling; the same reasoning §8.4 applies to the agent gate	—
R-111	The Day-0 artifact channel is 17 agent releases stale — a box installed today gets agent 0.96.0, not 0.113.0. felhom-host-install.sh does not use main: it reads the hub-vouched manifest (:423-436) and fetches Gitea generic packages (agent :1945, golden :2573). Gitea's newest are agent 0.96.0 and golden 0.161.0, and the hub's manifest selects exactly those — so a fresh box lands on agent 0.96.0 + controller 0.161.0 (global floor v0.156.0 < the golden's 0.161.0, so no self-update) against main's 0.113.0 / 0.185.1. Agent 0.113.0 reached both demo boxes by direct deploy and was never published	SHIPPED 2026-07-29 — the channel now serves agent 0.113.0 + golden 0.185.1	—	FIXED the same day it was found. Agent 0.113.0 built from the clean tree @ 58b598b and published (scripts/publish-agent.sh), sha 5f3247f756cb658e…, round-trip GET verified. Golden 0.185.1 baked on the nested drill VM embedding controller 0.185.1, published, sha dba00f3e845c415e… — bake clean: Result=success, overlay2, all 3 mounts included (rootfs+mp0+mp1), 0 FATAL/exclusions, upload HTTP 201, token-leak grep 0; log drill/bake-0.185.1.log; GL-1 teardown done (guest 9100 purged, secrets shredded, disk restored to virgin). Hub Day-0 manifest moved both together in one POST so it never vouched a new agent against an old golden; min_agent 0.93.0 → 0.113.0, which is what controller v0.185.0 declares (felhom-controller/CHANGELOG.md:15) — zero fleet impact, verified: all three enrolled hosts already run agent 0.113.0, so no box is held. wrapper_sha256 preserved verbatim (re-checked against configs/felhom-pbs-apply — no drift). The global controller floor was deliberately NOT raised: the golden now bakes 0.185.1, so a fresh box needs no self-update, and raising it would have been an unnecessary fleet-wide write. Original finding follows. Found 2026-07-29 by the E-2d Phase 0 gate, which stopped the run before a VM was created. 17 unpublished releases (v0.97.0–v0.113.0) strand the entire R-82 tiered-backup arc plus F-CRIT-2 (a failed backup looking fresh — 7 days silent) and F-REBOOT (a guest rebooted mid-backup never returns): a new customer's box would install without them. Blocks E-2d's C3/C4/C5 — those test endpoints and events that do not exist in 0.96.0/0.161.0. The controller is fine (registry has 0.185.1, floor-driven self-update), so the gap is specific to the two Gitea-generic artifacts. Mirror of R-110, not a duplicate: R-110 = the installer publishes instantly with no staging; R-111 = the agent/golden publish gate exists and was never walked. Fix should decide whether publishing joins the release train rather than staying a remembered step (R-29's shape, one layer up). Evidence: audits/E2D-fresh-vm-2026-07-29.md DEFERRED LEG, AND IT RECURRED → R-115. This row's shipped half stands and is not reopened: the bump happened, was verified, and was proven end-to-end by the E-2d install. But its own closing line — decide whether publishing joins the release train rather than staying a remembered step — was never acted on, and agent 0.114.0 reproduced the exact condition the same afternoon. The recurrence is filed as R-115, not as a reopen, because the stale-channel finding is closed while the process defect that caused it is a distinct problem with a distinct fix.	CC
R-115	Publishing is a remembered step, and it was forgotten within eight hours of being documented as forgettable. A box installs the agent from a Gitea generic package the hub explicitly vouches, never from git. Nothing in the build, deploy or session-end path publishes or checks that a version was published — so "deployed" and "installable" are independent states that drift silently. Two instances, both real: R-111 (2026-07-29 morning) — 17 agent releases v0.97.0–v0.113.0 stranded, so a new customer would have installed without the entire R-82 tiered-backup arc plus F-CRIT-2 and F-REBOOT; found only because the E-2d Phase 0 gate happened to look. Agent 0.114.0 (same afternoon) — the R-113 fix, built and pushed at b58d7bc, deployed to felhom-pve, and unpublished until this task, which blocked Session C: a fresh drill box would have installed 0.113.0 and proven the bug rather than the fix	CLOSED — SHIPPED (release-agent.sh + check-published-versions.py, 2026-08-03)	—	The finding is the RECURRENCE, not either instance — both instances are fixed. R-111's own text already named this leg ("decide whether publishing joins the release train rather than staying a remembered step") and closed SHIPPED without it; the leg then recurred the same day, which is the evidence that a note is not a mechanism. Class: → R-29, one layer up — a control that exists and is never walked; deliberately NOT given its own ID. The decision is the operator's; the options, mechanisms first: (a) publish as a step in the build/release path, so deployed and installable cannot diverge; (b) a gate that refuses to deploy a version that is not published+vouched — the strongest, and it fails closed; (c) a session-end checklist entry; (d) accept it as manual and add a pre-Session-C verification. (a) and (b) are mechanisms; (c) and (d) are reminders — and R-29's whole finding is that reminders do not hold. No code this session by design. THIRD INSTANCE, 2026-08-03 — and it was found by a runbook that had been told there was nothing left to do. Agent v0.120.0 — the agent half of the R-165 merge — was built, committed at cd6e267, and deployed to BOTH demo hosts, and was never published: GET …/generic/felhom-agent/0.120.0/felhom-agent → HTTP 404 (0.119.0 → 200), and the hub manifest accordingly vouched 0.119.0. The consequence is the sharpest yet, because installer step 5's idempotent skip requires installed == vouched EXACTLY: a documented-path reinstall would have downgraded both boxes from the merge-aware 0.120.0 to the pre-merge 0.119.0 — silently, since the current step_grows sets SYSDATA_GROW=0 so 0.119.0's mp1 resize (bringup.go 4c, fatal on error) never fires and the install would have succeeded while proving a stack nobody ships. R-178's own row asserted agent v0.120.0 is live on BOTH hosts and no code left to write; both were true and both were beside the point — the gap was publication, which no one checks. Fixed in-session on the operator's ruling: scripts/publish-agent.sh 0.120.0 (sha a7763d31b55b5ce75457b4dba7b06aa300325811834b0be78af4587b47110b9d, round-trip GET verified) then vouched, and both reinstalls then fetched and sha-verified it from Gitea. This is the third instance of a row that has been WAITING-ON-OPERATOR since 2026-07-29; option (b) — a gate that refuses to deploy or vouch an unpublished version — would have caught all three. OPERATOR RULING 2026-08-03 — mechanism (b), build-side: a gate that REFUSES to deploy or vouch a version that is not published. It is the strongest of the four options and the only one that fails closed; (c) and (d) were reminders, and R-29's whole finding is that reminders do not hold. Recorded here, not built this session, by instruction; it is now CC's to build. The third instance is the argument for the ruling and belongs inside it: agent v0.120.0 (the agent half of the R-165 merge) was built, committed and deployed to BOTH demo hosts while GET …/generic/felhom-agent/0.120.0/felhom-agent returned HTTP 404, so the hub vouched 0.119.0. Because installer step 5's idempotent skip requires installed == vouched EXACTLY, a documented-path reinstall would have silently downgraded both boxes to the pre-merge agent — and would have succeeded while doing it, since the current step_grows sets SYSDATA_GROW=0 so 0.119.0's fatal mp1 resize never fires. A gate at deploy/vouch time is the only one of the four standing between that and the operator. CLOSED 2026-08-03 — both halves, no version bump (no Go code changed). (1) scripts/release-agent.sh is now THE release path: build → tag → publish → verify by an INDEPENDENT download. It calls the existing publish-agent.sh rather than reimplementing it, refuses a dirty or unpushed tree, refuses to re-release an existing version (one version name must never mean two binaries), and deliberately does not vouch — vouching points machines at a version and stays the operator's act. CLAUDE.md's raw go build line is replaced by it, so the documented way to release cannot complete without publishing. It also tags, because R-183 made the tag part of the released artifact. (2) scripts/check-published-versions.py, registered in agent_gates.py as not --fast — it needs network, and a push must not fail because Gitea blinked. The CI workflow now runs the FULL set instead of --fast, without which the gate would have been registered and never run: the built-but-never-wired failure this project has shipped four times. THE INVARIANT IS NOT THE ONE THE TASK SPECIFIED, and the reason was measured (P-C), not argued. §8.4 asked for "the version the hub tells machines to install must be downloadable" — the better invariant, and CI cannot see it: the hub's artifact manifest is 401 without a per-customer passphrase and Gitea's package listing api is 401 without a token, while the package download url and the tags api are anonymous. Adding an operator credential to CI is the operator's call, not a gate author's. The implemented invariant — every v<semver> tag must have a downloadable package and a tag tree serving the agent's configs — needs no credential and catches all three recorded instances, because the release script creates the tag and publishes in one act. What it does NOT catch is stated rather than assumed away: the hub vouching a version that was never released at all → R-184. Red-proof F, MEASURED ON REAL CI and not inferred: runs 69 and 70 are on the same commit 0db7766 — success before a tagged-but-unpublished v9.9.9 existed, failure after pushing it. Same code, same workflow, one variable. Locally: the gate exits 1 naming the 404; deregistered from the entry point the same bad state reports all agent gates OK rc=0; restored → CONVICTED: published rc=1. v9.9.9 deleted afterwards (git ls-remote --tags → only v0.120.0). One deliberate CI failure e-mail reached the operator at ~12:5x CEST — that was this proof, not an incident	—
R-116	The drive-absent alarm and its recovery were a MISMATCHED PAIR — absent fired the GENERIC storage_disconnected, return the SPECIFIC backup_target_restored; backup_target_absent never fired at all	SHIPPED + PROVEN-LIVE (agent v0.116.0, 2026-07-30)	—	CLOSED. The full four-event sequence, on the wire, on a fresh box (audits/R116-v0116-2026-07-30.md): backup_target_absent (error) on detach → backup_target_restored (info) on return for the TARGET, and storage_disconnected (error) → storage_reconnected (info) for a NON-target drive on the same box four minutes apart. Two matched pairs, correctly discriminated — and discrimination is proven NON-trivially for the first time, since both prior runs had the target itself emit the generic event. Gate fired in 3 s; all four events reached the hub, so the specific alarm, its severity, its Hungarian copy and the hub routing are now exercised end-to-end. Over-correction PASSES with a positive observable (0 ABSENT lines / 0 drive events over 2m14s with both drives present, target degraded:false, while 2 RETURNED lines prove the gate was ticking). NARROWED by the R-117 spike (audits/SPIKE-r117-bind-liveness-2026-07-30.md §12), and it stands as written: the 2 RETURNED lines are a genuine positive observable, so rule 3 is satisfied — but degraded:false over that window was read off a drive whose bind was dead (R-117), so the window evidences "the gate did not over-fire" and NOT "the drive was healthy." No other part of this row changes: every input to the pairing fix is configuration-derived (storage.cfg's path vs the .mount unit's Where), which R-117 does not touch. Ran on a nested PVE on demo-hp per runbooks/target-selection.md — through the real day-0 from the v1.25.0 ISO, with the agent installed unaided from the vouched Day-0 manifest (published sha b47c5c4dab641ee5…, independent registry GET verified, manifest read back), drives enrolled through the real endpoints, device loss a real hot-detach. THE FIX, and the ruling is the substantive part: the mechanism was first isolated from the captured payload (DIAG-r116-disks-payload-2026-07-30.md) after two fixes aimed at shapes that do not occur. Both smaller-looking options were REJECTED because they regress R-114 — backup_target_offer.go:79 reads BackupTarget && MountPath != "" as "a real drive with its own mountpoint — healthy" and returns before its TargetAbsent branch, so back-filling MountPath on the Observe row or flagging the registry row (whose MountPath is the stale unit-file value) would have told the customer the backup target is fine while its drive was gone. R-114's correctness was resting on R-116's bug — a coupling invisible until the payload existed. Taken instead: the Observe row gets the guest path only (mount_path stays "", which is true) from a new ConfigPath (json:"-", so the cross-repo golden + key-set contract is untouched), and the union row is deduped on guest path — the join being CONFIGURATION (storage.cfg's path vs the .mount unit's Where), the only identity that survives the device. Tests 845→849; 4 red-proofs each asserted to land, and red-proof 1 replays v0.115.0's code and fails, which is the empirical proof it was inert. Its green test had supplied a MountPath production never supplies AND left DriveTargets nil so the union loop never ran — both corrected. v0.115.0 left in place (inert, harmless). Teardown all 3 layers; hub layer gate-blocked on ONLINE with the command recorded. Caveat: the drill's controller was 0.185.1 from the golden, which PREDATES R-114, so its absent-state banner showed the old false copy — the golden being a release behind, not a regression → R-120	—
R-120	The golden baked a controller that predated R-114 + R-112, so a FRESH box showed the customer the WRONG absent-target message	CLOSED — golden rebaked + PROVEN-LIVE, and the class now has an ENFORCED gate (golden 0.186.0 + hub v0.82.0, 2026-07-30)	—	audits/R120-golden-rebake-2026-07-30.md. Half 1 — the artifact. Golden 0.186.0 baked from main's controller in the DooPlex bake fixture (overlay2 OK, 3 mounts, FATAL 0, exclusions 0, 618 MB, upload 201, GOLDEN_SHA256=b760ac6a33e70700…, token-leak grep 0, GL-1 teardown, drill.qcow2 back to virgin). Three observables: published — anonymous GET (what the installer does) 200 / 648930639 bytes / sha identical to the bake; vouched — manifest read BACK; resolved — Artifact manifest served for customer sess-f (agent=0.116.0 golden=0.186.0). Floor untouched per publish-train rule 2 (min_controller_version still 0.156.0; it is a separate form); MinAgent left 0.113.0 as 0.186.0 declares. Proven on a REAL day-0, not the fixture (per the Part-1 rule now in runbooks/target-selection.md): VM 9402 on demo-hp from the v1.25.0 ISO → Controller elindult (0.186.0). With the target detached the endpoint returned the TargetAbsent copy — „A rendszermentés meghajtója nem érhető el — amíg vissza nem csatlakoztatod…" — and offer_path absent entirely; the day-old read on the 0.185.1 golden had returned the false system-disk message plus an offer of the other drive. Half 2 — the mechanism, operator ruling REFUSE. hub v0.82.0: the gate sits in hub/internal/web/configs.go handleSetArtifacts immediately before the only write — the sole UI path to SetArtifactManifest — so it runs on every vouch without anyone choosing to, and it refuses rather than warning. Signal: store.NewestReportedControllerVersion() over reports.controller_version, semver-compared in Go (MAX() in SQL ranks 0.99.0 above 0.186.0 — a pair this fleet has shipped). Fail-open in exactly two deliberate cases: empty golden field, unknown fleet version. NEAR-MISS RECORDED: the first draft read guests.controller_version, a column that exists and that NOTHING writes — it would always have seen "" and failed open, i.e. inert, this gate's own failure shape, one grep from shipping. 4 tests through the production handler over httptest (never a seam), the refusal asserting both the flash and that the manifest was not written; red-proof: deleting the block makes the stale golden vouchable again. PROVEN LIVE on the deployed hub by re-attempting the original mistake: vouching 0.185.1 → HTTP 303 …flash=golden_behind_fleet + [WARN] artifact vouch REFUSED: golden 0.185.1 is older than the newest controller the fleet reports (0.186.0), and the manifest read back unchanged at 0.186.0. Recorded on R-29's audit list (ROADMAP.md) as the first enforced gate beside its three orphans, so the contrast is kept — the orphans are unchanged. Teardown all 3 layers; hub layer gate-blocked on ONLINE with the command recorded, exactly as sess-e was (and sess-e was deleted this run)	—
R-117	A drive's guest bind becomes a DEAD MOUNT while every signal reads healthy — and it happens in TWO ways, only one of which the original framing covered. (a) after a detach/return: the host raw mount heals onto the NEW device via its fs-UUID-keyed unit while the bind still names the OLD one, so the gate takes its Return branch and restarts the customer's apps onto a namespace that EIOs on every call; (b) in STEADY STATE, no cycle at all — a device that errors without disappearing leaves the raw mount active, BoundUnderParent true and the drive never Disconnected, so the gate produces no action and NOTHING is emitted on any channel	SHIPPED + PROVEN-LIVE (agent v0.117.0, 2026-07-30)	—	CLOSED. audits/R117-v0117-2026-07-30.md. BoundUnderParent gains a THIRD term at both /disks sites: bindLiveness reads /proc only and requires (a) the bind names the same device as the raw mount and (b) the filesystem has not aborted (shutdown or emergency_ro, both measured). BOTH CHECKS ARE LOAD-BEARING and this is the substantive part: R-117 was filed as a detach/return defect, but a device that fails WITHOUT disappearing gives the identical all-signals-healthy state with the devnos EQUAL and the drive never Disconnected, so the gate emits nothing at all, indefinitely (R-117a) — the device comparison alone cannot see it, and a P1-only fix passes every payload test (red-proof RP3 exists for exactly that). THREE states, never a bool: {Unknown, Live, StaleDevice, Aborted}, Unknown is the zero value, and every caller reads Usable() where unknown counts PRESENT (absent stops a customer's apps — the newestArchiveOn trap). NO NEW RECOVERY PATH: AttachDrive's normalize leg already did the repair and three call sites already invoked it (20 s ticker, agent startup, and the controller's Return branch BEFORE restartStacks); all three were defeated by if n == 1 && GuestSeesMount(...) logging "fully live, no-op" about an EIO namespace. RULING (asked for, given, flagged for overrule): StaleDevice ⇒ re-bind (the raw mount is a healthy new superblock; repairs live, guest never restarts — init PID identical); Aborted ⇒ quiet no-op and SURFACE, because a re-bind lands on the SAME dead superblock and this runs every 20 s = an infinite silent retry that masks the state. No operator decision required: it routes an already-broken state into the existing gate, event types and Hungarian copy — no new customer-facing concept — and the alternative is apps writing documents into a filesystem that rejects every write. ORDERING TRAP caught by a test: abort-first classifies the real return state as aborted (its stale bind carries shutdown too) and refuses the repair while still reporting correctly, so the abort flag is read off the RAW mount in the stale case. LIVE on demo-hp (brought 0.113.0 → 0.117.0 first — see R-121): RETURN raw 8:32 / bind 8:16 shutdown ⇒ stale-device, usable false; IN-PLACE both 252:11 emergency_ro, raw unit still active ⇒ filesystem-aborted, usable false; healthy ⇒ live; 340–497 µs. No block I/O proven by strace (only /proc/self/mountinfo, 0 statfs) — the Part 1 CLAUDE.md fence applied to its own first consumer. No regression through the REAL pipeline: GET /disks with the controller's own credential shows the live backup-target drive bound_under_parent=True, with 32 gate lines in 3 min as the positive observable and zero spurious transitions. Tests 849→863, 29/29 green, 6 red-proofs each verified to land — and RP1 failing to fail exposed a HOLLOW test: the aborted fixture used a /dev/mapper device, for which RoleForStorage derives role=system, and a system row never runs the conjunction, so it reported false by DEFAULT and no mutation could fail it. Fixtures now assert the production row shape first. Teardown all 3 layers; hub layer = the vouched manifest, retained (it is the product, not scratch). NOT covered: the stale-bind repair on hardware — StablePathForRaw hardcodes the live parent, so it would write into guest 9201's namespace (R-117h); and sustained-load behaviour, still unmeasured. Follow-ups R-117g (no guided recovery for an aborted fs), R-117h (parent dir not test-seamable), R-121	CC
R-121	A BOX's installed agent can sit releases behind the vouched one and nothing notices — the R-120 gate does not cover it. demo-hp ran agent 0.113.0 while the hub vouched 0.116.0, through the whole R-116/R-117 arc, and no signal existed on any channel	READY (S) — NEW 2026-07-30	—	Fourth instance of the drift family (R-111 golden's agent 17 releases behind, R-115 built+deployed but never published, R-120 golden a controller behind — and now installed-vs-vouched on a live box). Confirmed at source that R-120's gate cannot catch it: hub/internal/web/configs.go:1165-1169 compares goldenVer against store.NewestReportedControllerVersion() — it is a golden-artifact vs fleet-CONTROLLER check and says nothing about the agent installed on a box. MinAgent does not cover it either: it is used to HOLD the controller floor for a box whose agent is too old (hub/internal/api/handler.go:530-538, store.go:1857) — protective, not an alarm — and demo-hp's 0.113.0 equalled min_agent 0.113.0, so even a floor comparison was satisfied. The cost, measured: R-117's whole subject is the R-113 conjunction, which landed in 0.114.0 — so the designated drill host could not exercise the code under investigation at all, and the R-117 spike had to route every predicate result through an out-of-repo probe built from main instead of the installed agent (audits/SPIKE-r117-bind-liveness-2026-07-30.md §1, §2.3). Discovered because the R-117 task made bringing the box current an explicit prerequisite. Fix shape (not implemented): the hub already receives AgentVersion on every host report, and already has semver comparison in Go — the missing piece is a checker comparing reported agent vs the vouched agent and surfacing it, operator-tier. Note the honest tension: a box legitimately lags between publish and deploy, so this wants a staleness window rather than an instant alarm	CC
R-118	An absent drive's union row advertises the ROOT filesystem's capacity as its own. In the absent-state payload the registry-union row reports total_bytes: 49675956224 / used_bytes: 4584579072 — byte-identical to the local row (durable_id: path:/var/lib/vz, i.e. pve-root) in the same response. The real drive is 4 GB	READY (XS) — NEW 2026-07-30	—	Cause: statfsCapacity(d.MountPath) (disks.go:335-338) statfs's /mnt/cel, which with the device gone is a bare directory on the root filesystem. observe.go:176-183's comment warns about exactly this trap and guards the Observe path ("an unmounted removable dir-storage's mountpoint reverts to a bare directory on root … catastrophic DR mis-id"); the union path has no equivalent guard. Not a DR mis-id — durable_id on that row is still the correct uuid:…, so re-attach identity is safe. It is a false capacity reaching every consumer of total_bytes/used_fraction (fill monitors, storage cards): a detached 4 GB drive advertises 46 GiB at 9.2 % used. Same class as role.go:180-181 — an absent drive's fields decaying to the root filesystem's. Evidence: audits/DIAG-r116-disks-payload-2026-07-30.md §12	CC
R-113	The drive-absent gate CANNOT FIRE on device loss — E-2b's alarm is wired to an unreachable condition. planDriveGates (intermediary.go:216-262) computes presence by OR-ing d.BoundUnderParent into present[GuestPath], and the agent derives BoundUnderParent from GuestSeesMount() — "is this path a mount target in the guest's /proc/<pid>/mountinfo" (localapi/disks.go:210). The raw drive mount is a device-bound systemd unit and dies with the device; the agent's own bind under the shared parent is NOT device-bound, so its mountinfo entry outlives the device. The gate reads the surviving bind as "present" ⇒ no Stop action ⇒ notifyDriveAbsent never called	SHIPPED + PROVEN-LIVE (agent v0.114.0, 2026-07-29)	—	PROVEN LIVE audits/SESSION-C-2026-07-29.md. Target drive hot-detached on a fresh box running the SHIPPED agent 0.114.0 (from the Day-0 manifest, not a hand build). The gate fired in 4 seconds — E-2d measured ZERO over 4½ minutes — and SetDisconnected was reached. It fired on exactly the shape that defeated it: raw /mnt/mentes NOT mounted while the bind /mnt/felhom-drives/mentes still read /dev/sdb[/felhom-data]. Over-correction guard PASSED: with both drives present, 0 ABSENT lines and the target stayed healthy. Note: C5 still FAILED — but on a different defect (→ R-116), not on this one. SESSION C IS NOW UNBLOCKED. Agent 0.114.0 published and vouched 2026-07-29 (sha 5e4c15ebee2d7583…, round-trip GET verified; hub manifest read back showing it) — a fresh drill box now installs the fix under test instead of the bug. The golden was NOT re-baked and min_agent was NOT raised: the golden bakes the controller, not the agent, and controller v0.185.0 declares MinAgent 0.113.0, which 0.114.0 already satisfies. FIX: BoundUnderParent is now a CONJUNCTION — bound under the parent AND the drive's raw host mount still mounted (devicePresent, new deviceCheck seam), at BOTH /disks construction sites. The raw mount is the device-bound systemd unit that dies with the device; the agent's bind is not — so the raw mount IS the device signal, grounded in E-2d's measurement rather than inference. Conjunction, deliberately: the device half alone would regress boot ordering (raw mounts early, bind lands ~18 s later — that window must keep reading absent), so existing behaviour is byte-identical and only the unreachable case is closed. Unknown is never absent (devicePresent("") = true) — a false absent stops a working customer's apps. Controller UNCHANGED, no MinAgent bump: BoundUnderParent has exactly one functional consumer (planDriveGates:226); a new DevicePresent bool was rejected because absent-from-JSON decodes to false, which would have made every drive on an older agent read ABSENT. +6 tests (208→214), 4 red-proofs run and reverted. Deployed to demo-felhom; over-correction guard verified in production — raw mount present, drive still reads present, 10/10 apps untouched, no gate action, no false alarm. demo-hp deliberately left on 0.113.0 (the spec scoped deploy to felhom-pve) — it still carries the bug. ⚠️ SESSION C BLOCKER: the hub Day-0 manifest vouches agent 0.113.0, so a fresh drill box would install WITHOUT this fix and validate nothing — publish + vouch 0.114.0 first (R-111's trap, same shape). Awaiting live proof of exactly one leg: device loss → gate Stop → SetDisconnected → backup_target_absent on the wire. Original finding: PROVEN LIVE 2026-07-29 on a fresh box. Target drive hot-detached under a running agent; over 4½ min (budget was 60 s): the agent said enrolled drive absent by UUID every 20 s, the controller logged 0 [gate] lines, and the hub received zero events — no backup_target_absent and no generic storage_disconnected. Measured with the device gone: /mnt/mentes2 NOT mounted, /mnt/felhom-drives/mentes2 still /dev/sdb[/felhom-data]. Not a virtualisation artefact — the asymmetry is device-bound-mount vs manual-bind, identical on metal (caveat: proven on SCSI hot-detach; physical unplug not staged). Mirror scenario reasoned, not observed: both the specific and generic events come from the same a.Stop branch, so the generic one is equally unreachable. Sixth instance of seam-built-but-never-wired — E-2b wired the seam, to a condition that cannot occur. Evidence: audits/E2D-fresh-vm-2026-07-29.md §5.2 Session C scope UNCHANGED by Session B — R-114/R-112 shipped as controller v0.186.0 and neither touches the agent; the leg awaiting proof is still device loss → gate Stop → SetDisconnected → backup_target_absent on the wire. One rebuild now validates all three.	CC
R-112	E-2's degraded banner and offer have NO UI CONSUMER — the endpoint is correct and the customer never sees it. GET /api/storage/backup-target returns byte-exact copy (proven live), and nothing fetches it: grep 'backup-target' across all *.html/*.js/*.css → 0 hits; no template references OfferPath/Degraded/the copy; resolveBackupTargetState + degradedMessageFor are consumed only by the JSON handler — no page handler injects the state	SHIPPED + PROVEN-LIVE (controller v0.186.0, 2026-07-29)	—	PROVEN LIVE audits/SESSION-C-2026-07-29.md — the banner reached a customer's page for the first time. Never-configured box: banner element 1, never-configured copy 1. After the wizard: offer block 1 with data-path="/mnt/felhom-drives/mentes". Healthy after assign: all four markers 0, proven POSITIVELY — idle delta 0 /backup/tiers calls, page-load delta +1, single caller ⇒ the seam ran and chose silence. FIXED: the state now has a consumer. Server-rendered on /backups via backupsHandler → backupTargetView → backups.html, following the existing SingleCopyWarning banner pattern — NOT a 19th JS fetch, because a banner that needs JavaScript to appear is one more thing that can silently not happen. backupTargetView returns nil for healthy AND unknown, so those render nothing at all. Scenario-E seam test drives backupsHandler over httptest and asserts the RENDERED HTML — deleting the one line that sets data["BackupTarget"] reproduces the old state and fails every render assertion. SEAM PROVEN LIVE on demo-felhom by a DIFFERENTIAL positive observable, not by an absent banner: idle 8 s → 0 new /backup/tiers agent calls; each /backups load → exactly +1, and that call has only one caller (resolveBackupTargetState). The box is healthy (degraded:false, target:felhom-backup) and the page correctly rendered nothing — which matches its real state but, being a negative, is NOT by itself proof of wiring. Still unproven live: that a customer sees actual copy — impossible on a healthy box. The decisive contrast: templates fetch 18 distinct /api/storage/* endpoints; backup-target and backup-target/assign are the only two with zero references. The handler's own comment calls itself "the dashboard's source for the degraded banner and the offer" — an invariant comment asserting a consumer that does not exist (7th instance of that class). v0.185.1 shipped as "the offer endpoints were mounted where nothing routed to them" — it fixed the router mount and stopped one layer short of the render; its test TestBackupTargetRoutesLiveUnderTheStorageAPIMount pins dispatch, not reachability, which is exactly what CLAUDE.md's seam rule warns about. Fifth instance of seam-built-but-never-wired. Fix R-114 FIRST — wiring this alone would start showing customers the wrong message. Evidence: audits/E2D-fresh-vm-2026-07-29.md §5.1	CC
R-114	On target-drive loss the customer is told the wrong story and offered the drive that just vanished. With the assigned target absent, the endpoint returned degraded:true, target:"felhom-backup" plus the "a rendszermentés ugyanazon a lemezen van, mint a rendszer" message — false, the target is a drive that has disappeared, not the system disk — and offer_path pointing at the missing drive as the remedy	SHIPPED + PROVEN-LIVE (controller v0.186.0, 2026-07-29)	—	PROVEN LIVE audits/SESSION-C-2026-07-29.md. With the target absent the page rendered the ABSENT copy (1), the system-disk copy 0, the offer block 0 — both of E-2d's falsehoods gone. API carried message:"A rendszermentés meghajtója nem érhető el…" with target:felhom-backup. FIXED: the third state exists. New BackupTargetState.TargetAbsent separates configured-and-gone from never-configured. Degraded keeps its meaning (is there a problem) so the wire contract is unchanged for every consumer; TargetAbsent answers which problem, because the remedies are OPPOSITE — attach any second drive vs reconnect that one. Copy routed through degradedMessageFor (still one decision point) and taken verbatim from the hub's backup_target_absent email so banner and mail tell one story. Offer suppressed on the branch itself, deliberately not left to firstOfferableDrive's Disconnected skip — that flag is set by R-113 in another repo, and this state must be right without it. Red-proof: deleting the branch reproduces E-2d's exact payload, offering /mnt/felhom-drives/mentes2, the drive that had vanished. MinAgent unchanged 0.113.0 — R-114 reads BackupTarget/MountPath/GuestPath/Role, none of which R-113 altered, so demo-hp is not held. NOT live-validated: Scenario C cannot occur on a healthy box. resolveBackupTargetState falls through to the generic degraded branch whenever no disk satisfies d.BackupTarget && d.MountPath != "", never distinguishing never configured from configured and now missing. Shares R-113's root cause — two disagreeing presence signals — but is a different code path with a different fix. Currently invisible ONLY because of R-112; fix this before wiring that. Also seen: after reattach the drive returned as /dev/sdc while the stable bind still recorded /dev/sdb, and the state read healthy. Evidence: audits/E2D-fresh-vm-2026-07-29.md §5.3	CC
R-29	The green gates are not enforced anywhere — one was RED for 16 releases before anyone ran it. This is the class, not an instance: a gate that exists, asserts something true, is red, and is invoked by nothing reads as coverage it is not providing. controller/scripts/docker_run_volume_path_gate.py failed continuously from 2026-07-14 (v0.129.0) until R-7b's close-out ran it by hand at v0.145.0 — sixteen releases in which every REPORT said "green"	CLOSED — both halves shipped (2026-08-02)	—	This item has existed at ROADMAP.md:158 since before the register was rebuilt (2026-07-27) and was never carried across — that omission is itself part of the finding, because it is an open item about work not getting done that then went missing from the page that decides what gets done. Two separable parts, per R-29's own analysis: (a) the docker_run_volume_path_gate finding is benign and the fix is a 3-line ALLOWLIST addition with its why — not a rewrite of the flagged call — and it gets its own reviewed diff, never bundled into a feature commit; (b) the systemic half, the real item: decide where gates run (pre-push hook, build.sh step, or CI) and make a red gate block the train the way the Go green gate does. Two further orphans confirmed 2026-07-29 by repo-wide grep across all file types + sibling repos + ~/.claude settings/skills/hooks + .git/hooks (none non-sample) + Makefile/justfile/Taskfile find (only hub/Makefile, zero gate occurrences) + CI-directory find (this repo has no CI at all) — every one of the 19 hits is a docstring, a code comment or prose, and not one is an invocation: scripts/hostinstall_gates.py — RED today (hub Setup-tab hostInstallVersion=1.19.0 != SCRIPT_VERSION=1.22.0, exit 1), the same finding as R-94 leg (b) — and scripts/hub_confirm_gate.py. Of the four gates in scripts/, only site_gates.py is mandated anywhere (CLAUDE.md:153) and manifest_bearer_gate.py is named in runbooks/secrets.md:76. In R-29's own words, carried forward deliberately: do not mint a new ID for a new instance — the 2026-07-18 rehearsal independently re-raised this item and no second ID was minted then either UPDATE 2026-08-02 — leg (a) CLOSED (felhom-controller c432f70, its own reviewed diff as specified): appexport/estimate.go's -v is a NAMED VOLUME mounted read-only into a throwaway container, no host path, structurally identical to the allowlisted backup/backup.go entry — allowlisted with its why; realVolumeSize untouched. Leg (b) HALF-SHIPPED: the 'decide where gates run' ruling is now made and half-implemented — every repo has ONE entry point (felhom.eu/scripts/repo_gates.py, felhom-controller/controller/scripts/controller_gates.py, felhom-agent/scripts/agent_gates.py, app-catalog-felhom.eu/scripts/catalog_gates.py), each mandated in its CLAUDE.md and each wired to .githooks/pre-push via --fast. THE CENSUS, which is the finding: thirteen gate scripts across four repos; every gate a CLAUDE.md names was GREEN, and two of the four nobody names were RED — hostinstall_gates.py (red since 2026-07-14) and reuse_refs_check.py (red on all four repos); a third, docker_run_volume_path_gate.py, was named only in REUSE.md:284 and was also red. Correlation with 'named in a CLAUDE.md' was exact. STAYS OPEN for the automatic half — a hook is per-clone and --no-verify skips it; the unbypassable half is CI → R-168 CLOSED 2026-08-02, on the demonstrated ALARM and not on a green run. Leg (b)'s automatic half is now live: a Gitea Actions runner re-runs every repo's entry point on every push, independent of who pushed and of what they typed (→ R-168). The class this row opened — a gate that exists, asserts something true, is red, and is invoked by nothing — is answered at both ends: the pre-push hook refuses locally, and CI catches a --no-verify bypass and emails the operator, proven with a real red run and a provider accepted-id. What remains is not this row's finding but a working-style choice — CI reports rather than blocks because there is no merge to gate (→ R-169)	CC
R-95	restic offsite credential can delete (readonly=False, forget --prune runs from the box); SFTP cannot express append-only	READY	—	Root exposure still open. Mitigation now ARMED — split prune off-box or move to REST --append-only	CC
R-86	Restore-tests are interval-scheduled, not backup-aligned	CLOSED — SHIPPED + PROVEN-LIVE 2026-08-03 (agent v0.121.0, hub v0.91.0)	—	The rule that shipped: let A be the newest archive on a tier that has settled ≥24 h; the tier is DUE when A exists and A has not already been proven. Daily tier → proved daily on yesterday's archive; weekly tier → weekly on its own; newborn → UNKNOWN. The daemon-start ticker survives only as the evaluation interval. THE TRAP, recorded because it is the version a reasonable person writes: the row's own wording implemented literally — "due when the newest archive is ≥24 h old" — is NEVER true on a daily tier, because a new archive resets the newest-archive age to zero long before it reaches the lag; it would have silently switched restore-testing OFF for the tier that matters most. Red-proved at 0 runs over 5 simulated days. The state now records WHICH archive was proven, not when a tier last passed — a time cannot answer have we proven this archive. A pre-R-86 state file keeps its time (rotation ordering survives) and yields no proven archive, so each tier is due exactly once after the upgrade: the safe direction. Two knobs replace one and the old one is not silently repurposed: restore_test_eval_interval_seconds (6 h) and restore_test_settle_seconds (24 h); the deprecated restore_test_cadence_seconds keeps its DISABLE meaning verbatim, now seeds the settle lag, and the daemon WARNs once at start-up naming both. 6 h is bounded from both ends, not picked: MEASURED cost of one evaluation on demo-felhom — local dir storage 18 ms, PBS tier over the WAN to ep0 392 ms, both 430 ms — so cost is irrelevant; the CEILING is that a FAILING tier stays due, making the evaluation interval its retry interval for a multi-GB restore. Part 2 shipped WITH it and was not optional — see the hub half in this row's sibling text and 07-backup-architecture.md §3: restoreProvenStaleAfter was a flat 7 days derived from the very cadence this removed, and a healthy weekly tier's proof age reaches exactly 168 h against a 168 h window — it sat ON the line, so any ordinary delay tipped it into a nightly alarm about a working system. The window is now per tier from that tier's observed archive interval, ×4 generations, floored at the old 7 days and capped at 12 days (strictly inside the two-week offsite retention), falling back to the tier's DECLARED rhythm (26 h host / 8 d offsite — the backup-freshness checker's own thresholds) when history is too short to observe one. A hollow test caught by its own red-proof: the first Scenario-G fixture had no jitter and PASSED under the flat-window mutation, because a perfectly regular weekly tier sits exactly ON the line rather than over it. The jitter is what makes it a test. Also fixed in passing: the candidate picker now skips archives failing archivePlausiblyComplete (under per-archive due-ness an incomplete phantom would be picked forever, fail forever, never earn proof and leave the tier due at EVERY evaluation), and the due-check runs BEFORE the heavy-operation gate is taken (a frequent poll must not be able to make a starting backup record a failure — F-A1). Live proof: see felhom-agent/REPORT.md	CC
R-87	The restic tier is never restore-tested	READY — RE-RANKED UP 2026-08-03 (R-86 closed)	—	Design a controller-side test (no scratch-guest analogue transfers). Most of what this row needed now exists. R-86 built the piece that was missing: a tier is proved per archive generation, on its own rhythm, with the proof recorded as which archive — which is exactly the shape a weekly-ish restic tier needs, and the reason this row could not simply reuse the whole-guest scheduler before. What remains is genuinely restic-specific and is NOT a scheduling problem: there is no scratch-guest analogue, so the test has to be a controller-side restore of a bounded sample into a throwaway path, with its own definition of "proved". Two things to carry over rather than re-derive: the proof must record the SNAPSHOT it proved (not a timestamp), and the hub's staleness window must learn this tier's rhythm the way restoreProvenWindow now does — a restic tier on a weekly cadence lands on the same false-alarm line the flat 7 days did. And R-95 still applies: that credential can delete, so a restic restore-test must never be able to write to the repo	CC
R-185	The agent cannot see the host backup tier's archives on demo-felhom — the PVE token has no ACL on /storage/felhom-backup, so the content listing returns EMPTY where root sees three archives. Found 2026-08-03 while live-validating R-86. pveum acl list grants FelhomAgentStore on /storage/{local,local-lvm,felhom-pbs} and not on felhom-backup, which is the box's actual local_backup_target. Verified three ways: pvesh as root lists 3 archives (6.1–6.3 GB, 08-01/02/03); the same endpoint with the agent's token returns {"data":[]}; and local — which HAS a grant — returns its archives through the same token	CLOSED — SHIPPED + PROVEN-LIVE 2026-08-03 (agent v0.123.0, installer 1.24.0)	—	Pre-existing and independent of R-86 (it is a property of the ACL, and the R-85 rotation had the same blindness). Consequences: the host tier has never been restore-testable on that box, and R-85's "an empty tier is skipped, not failed" rule made that silent. The part worth fixing is the silence, not only the grant: a permission-blinded tier is today INDISTINGUISHABLE from a newborn one — both report "no settled archive yet" — which is this project's own absence-is-not-evidence rule failing in a new place. The agent already knows better: it RECORDS successful backups to that target, so "I wrote archives here and the tier lists none" is a contradiction it can detect and should say loudly. Do not fix by widening the token blind: decide whether the host-install ACL set should follow local_backup_target (it currently hardcodes local), which is where the drift began BOTH HALVES FIXED, AND THE ROOT CAUSE WAS NOT WHERE IT LOOKED. (1) The silence. Client.Permissions asks GET /access/permissions?path=/storage/<target> as the agent's own token, and one capability.Status per configured tier reports it — composed around the sudo prober the way the pool-read check already is, with the Status wire shape untouched so the hub's existing critical-degraded alert applies with no hub change. MEASURED BEFORE IT WAS DESIGNED AGAINST, and the obvious reading is wrong: an ungranted path answers neither empty nor 403 but with the privileges INHERITED from the box-wide / grant — felhom-pbs → {Datastore.Allocate:1, Datastore.AllocateSpace:1} vs felhom-backup → {Sys.Audit:1, SDN.Use:1, Datastore.Audit:1}. A probe asking "is the path present?" or "does it have Datastore.Audit?" would report the blinded storage HEALTHY; the probe tests Datastore.AllocateSpace and a red-proof pins that. The probed set comes from the box's own BackupTiers(), never a fixed list — a hardcoded probe list is the defect reproduced inside the fix. Critical (the hub alerts only on critical), except the local fallback target, which is reported but does not page: turning host-install's own documented DEGRADED configuration into an alert is how a signal becomes something an operator archives unread. It never looks at content, so it cannot alarm on a newborn tier by construction; it never reports ok when it could not ask. (2) The installer. The row and the task both assumed PVE_STORAGES was the culprit. It is not: configure_backup_target's Case A arm creates the storage and grants in the same breath, so a box that builds its own target was always fine. The Scenario-F arm — "the target already exists, leave it exactly as it is" — returned without granting, which is why a box whose felhom-backup pre-dated the install (the vzdump-target-move runbook, or a reinstall) pointed local_backup_target at a storage its token could not read. That arm now ensures the ACL through the same guarded wrapper; the storage DEFINITION is still untouched, so Scenario F is unviolated, and pveum acl modify is idempotent. $BACKUP_TARGET_ID is deliberately still NOT in PVE_STORAGES — that list is granted in step 4/5, before the target is resolved in step 6, and --acl-storages entries are preflight-checked for existence. A gate now asserts every arm that resolves the target also grants on it (red-proved: resolves in 2 place(s) but grants in only 1). LIVE, in order: the three-way observation reproduced unchanged (root 3 archives / token {"data":[]} / local lists through the same token) → agent v0.123.0 deployed → capability DEGRADED … pve:store-grant:felhom-backup … lacks Datastore.AllocateSpace … INVISIBLE to the agent and it is never restore-tested, critical=true, and the hub raised agent_capability_degraded and emailed the operator → grant applied (user AND token) → the token lists 3 archives → ok=70 total=70 degraded=0 and the hub logged degraded → ok (agent_capability_recovered) → the host tier is a due-check candidate for the first time on that box, correctly picking the 08-02 archive because the 08-03 one has not settled 24 h. demo-hp CARRIES THE SAME DRIFT AND WAS FIXED TOO — local_backup_target=felhom-backup, no ACL row, now granted and listing 4 archives. §8.6 assumed one box was affected; the same one-line additive idempotent fix applies to the other, and leaving a known-blind backup tier on a Tier-0 box after finding it would be this row happening twice. The tester's box was NOT touched (Tier 2) — see the note in this row's outcome.	CC
R-186	A released agent binary's sha256 cannot be reproduced from its tag. release-agent.sh builds at step 3 and tags at step 4, so Go's VCS stamp records a PSEUDO-version (v0.120.1-0.20260803130452-4d825910…) in the published bytes, while any rebuild after the tag exists stamps v0.121.0 — a different binary. Measured 2026-08-03 on v0.121.0: published b2128f3c… (14 081 336 B) vs rebuild-at-tag 8302e396… (14 077 240 B), identical source, identical toolchain, 4 096 bytes apart	CLOSED — SHIPPED + MEASURED 2026-08-03 (agent v0.122.0)	—	Why it matters: the sha the operator vouches is the one thing tying a machine to a binary, and today nobody can independently rebuild it to check. The build order is deliberate (the script's own comment: a tag with no package is caught by check-published-versions.py, a package with no tag is invisible to it), so the fix is not to swap the steps blind. Candidates: -buildvcs=false or -trimpath for a version-stable stamp, or tag-then-build with the tag deleted on a failed publish. Mitigation used this session: the DEPLOYED binary is the PUBLISHED artifact, downloaded from Gitea — not a local rebuild — so the running bytes are the vouchable ones FIXED by removing the cause, not by sequencing around it. -buildvcs=false drops the VCS stamp (nothing in the repo reads it — no ReadBuildInfo caller) and -trimpath removes absolute build paths, so the same source produces the same bytes whether or not the tag exists yet and regardless of checkout directory. Measured before the change, same source, same toolchain, one commit: default flags with no tag 18f4a495… 14 085 464 B (mod v0.121.2-0.2026…) vs default flags tagged 4a38f394… 14 085 440 B (mod v0.121.99) — different bytes; with the new flags, 7ffcdf1d… 14 064 574 B identical both ways. Measured after the change on the real release: published d5f294e5… 14 076 649 B = rebuild-at-tag #1 = rebuild-at-tag #2, all three identical. A second discrepancy fell out of it: publish-agent.sh's fallback build forced CGO_ENABLED=0 and therefore produced a binary 74 KB smaller (13 990 236 B vs 14 064 574 B) than the release path built for the same version — one version name, two binaries, decided by which entry point ran. Both paths now build identically, with a comment in each naming the other. The operator can now check a published sha without being there: the exact command is in felhom-agent/CLAUDE.md.	CC
R-187	R-115's one-command release had never actually run its publish leg — the first real use died there. scripts/publish-agent.sh has been mode 0644 since it was created (2026-06-28), because every earlier caller invoked it as bash scripts/publish-agent.sh; release-agent.sh (written 2026-08-03) called it directly and got Permission denied on v0.121.0's release	CLOSED — SHIPPED 2026-08-03 (felhom-agent)	—	Fixed both ways in one commit: the executable bit restored, and the caller changed to bash "$REPO_ROOT/scripts/publish-agent.sh" so the release no longer depends on a file mode — the kind of thing a checkout, an archive or a copy silently loses again. The lesson is R-115's own, one level up: the mechanism written to make a step unforgettable was itself never exercised end-to-end, so it failed the first time it mattered. A mechanism that has not been RUN is a note with better formatting	CC
R-188	Every agent release has a ~50 % chance of emailing the operator a CI failure for a release that is correct. on: [push] fires the gates workflow on the tag push too, and release-agent.sh pushes the tag BEFORE publishing the package (deliberately — a tag with no package is caught by check-published-versions.py, a package with no tag is invisible to it). So CI can run the published-versions gate inside the seconds-long window where v<ver> exists and the package does not, and it correctly reports "every released agent version must be INSTALLABLE" — for a release that completes normally a moment later	CLOSED — SHIPPED 2026-08-03 (agent v0.122.0)	—	MEASURED, 2026-08-03, across two releases in one session: v0.121.0 → runs #12 (id 79) success and #13 (id 81) failure on the same sha 4618169; v0.121.1 → #17 (id 88) failure and #18 (id 89) success on the same sha cd1b087; and v0.121.0's second pair (#14/#15, 4d82591) both success — i.e. it is a race, not a rule. Local gate afterwards: all three versions verify. Why it matters more than it looks: R-168 made CI email the operator on failure precisely so a red gate cannot be missed. A signal that cries wolf on every second correct release is how that mail becomes something you archive unread — the R-29 failure mode one layer up, and R-100's corollary in a new place (an alarm whose text is true but whose trigger is not the thing you would act on). Fix shapes, cheapest first: have the workflow skip tag refs (if: !startsWith(github.ref, 'refs/tags/') — the commit push already covers the content, and the tag adds no new code); or make check-published-versions.py tolerate a tag younger than ~2 minutes; or publish before tagging, which the release script's own comment argues against. Do not "fix" it by quieting the alarm FIXED by moving ONLY the tag push (build → tag locally → publish → push tag). The tag is still created before anything is published, so the build and the tag still describe the same commit; it becomes VISIBLE — to CI, and to any raw/tag/… fetch — only once the package is downloadable. The invariant the old order protected was not traded away: check-published-versions.py now asserts the converse directly — no published version may be missing its tag — as a BOUNDED probe of the frontier (where a failed tag push leaves an orphan) and of patch gaps, printing its probe set on every run because a check whose coverage is invisible reads as a guarantee it is not making. The package listing api still answers 401 without a token (re-measured 2026-08-03), so absence still cannot be enumerated and the script says so. Both directions demonstrated: a real published-but-untagged fixture (0.121.2, published then deleted — HTTP 201 then 204, absence re-verified by a 404) made the gate FAIL and print the one-line recovery; removing the converse block made it pass over that same orphan. A tag with no package still fails, shown against a local stand-in rather than by pushing a probe tag — pushing one would have emailed the operator a TRUE alarm about a fixture, which is the same attention cost this row removes. A half-done release is now loud: publish-succeeded-then-push-failed dies naming the exact recovery command, and a FAILED publish removes the local-only tag so the release can simply be retried instead of colliding with step 2's re-release guard. Scenario E, live: v0.122.0's own release produced CI runs 21 (task id 96) and 22 (task id 97) on the same sha — both green, where the same shape produced one red run on each of the two previous releases.	CC
R-189	A passing restore-test can be invisible to the hub forever — and R-86 made that window a week instead of a day. The agent reports restore_tests[] from internal/backup.Store, which is in-memory (its own comment: "lost on restart; the cadence re-populates"), and the hub's report interval is 900 s. An agent restart inside that window drops the result. That used to self-heal within 24 h because the next cadence re-tested the same tier; under per-archive due-ness the agent will NOT re-test a proven archive, so the hub can stay ignorant of a proof until the NEXT archive generation — a week on the offsite tier. Observed live, 2026-08-03: the 15:25:14 PASS on felhom-pbs (a real 14.5 GB offsite restore) reached no host-report — the hub logged 0 restore-tests at 15:28:00 and 15:32:14 — because the agent was restarted at 15:27:57 for the v0.121.1 deploy, 2 m 43 s after the run and before the next report	CLOSED — SHIPPED + PROVEN-LIVE 2026-08-03 (agent v0.122.0)	—	The persisted proof already exists and is not reported. RestoreTestState (on disk, per tier, R-86: with the archive id) survives exactly this, and RestoreTestState.Snapshot() carries the doc comment "for the host-report gauge" — and has no production caller at all (grep, 2026-08-03). That is this project's own seam built but never wired class, fifth instance, and a comment asserting an invariant with no test pinning it, both in one method. Severity is bounded, and worth stating so it is not over-ranked: the hub scans its retained window, archives keep arriving, and R-86's per-tier window for the offsite tier is 12 days against a 7-day archive rhythm — so ONE lost report is tolerated. What is not tolerable is that the loss is silent and that the box will not repeat the work. Fix shape: report the persisted per-tier proof (target, archive, proven_at) alongside the in-memory latest run, and let the hub's lastProvenPerTier read it — a report-contract change, so it touches internal/hub/report.go + contract_test.go + the hub's checker, and it is a spec, not a one-liner FIXED. RestoreTestState now stores the tier and what was verified beside the archive (recorded at proof time, from the run's own result — deriving them later would need a storage lookup that can fail on the one path where failing means mislabelling a proof), exposes ProvenRestoreTests, and Collector.SetProvenRestoreTests merges it with the in-memory result. Merge rule: one entry per tier, newest by TestedAt wins — a fresh failure beats a stored success (the failure is the news and lives nowhere else), a stored success beats a stale in-memory entry after a restart, and a tier never appears twice (two entries would read at the hub as two tests). An unparseable timestamp counts as OLDER, so a malformed entry cannot displace a good one. It refuses to lie: a record missing the archive OR the tier produces NO entry, and run mechanics (scratch VMID, duration) are not re-invented — an absent duration is not a claim, a fabricated one would be. The asymmetry is now written where it will be read: only successes are persisted, because a success SUPPRESSES future work (a proven archive is never re-tested, so a lost proof leaves the box quietly less tested than it believes) while a failure CAUSES future work and heals itself at the next evaluation. Store's comment was corrected in place — "lost on restart; the cadence re-populates" was true under a timer and became false the moment R-86 shipped; leaving it is how the next reader concludes this is handled. The seam that had no caller is now pinned by an AST test (TestMainWiresTheDurableRestoreTestProof), because Snapshot() claimed a "host-report gauge" in its doc comment for weeks with nothing calling it. MIGRATION, stated because it is visible on the live box: a pre-R-189 record carries an archive but no tier and is therefore NOT reportable — upgrading does not retroactively make an old proof visible to the hub; the tier's next real proof fills it in. Observed on demo-felhom immediately after the upgrade: still 0 restore-tests until a fresh run.	CC
—	Storage Box snapshots on storage-box-pool-1 — plan SET (daily 00:00, keep 7) but 0 taken yet	WATCHING	first run tonight 00:00	Confirm size_snapshots > 0 tomorrow; until then the mitigation is armed, not proven	CC
—	PBS-storage-1 (u629193, box 611421) still status=active, 19.9 MB	WAITING-ON-OPERATOR	operator console	Delete the box	operator
R-90	ep0 RAM headroom — 4 GiB swap survived its first reboot 2026-07-27; 3.8 GB RAM unchanged	CLOSED — the operator rescaled ep0 to a CX33 on 2026-08-03	—	MEASURED ON THE BOX, not read from an invoice: felhom-hetzner reports Mem: 7757 MB total (8 GB, was 3.8) and nproc 4. The interim lever survived and was checked rather than assumed — a resize is a stop/start, so "the swapfile is still there" was an assumption until measured: /swapfile, 4 GiB, dated Jul 27 14:40, active (swapon --show → /swapfile file 4G 0B -2), 0 B in use on an idle box. THE 40 GB LOCAL DISK DID NOT CHANGE and must not be "corrected" alongside the RAM: / is 38 G, 58% used. This was a CPU/RAM resize only, so every disk figure in the runbooks still stands — the separate 98 G volume at /mnt/pbs-datastore (R-82 P0.3) is unaffected. Why this was BLOCKED and no longer is: the row recorded CX33 as "confirmed unavailable even powered OFF" — the Cost-Optimized line's limited availability, not a power-state problem. It became available and the operator took it. Documentation corrected (RUNBOOK-ep0-datastore-volume, RUNBOOK-pbs-prune-serverside ×2, runbooks/offsite-endpoint.md ×2, runbooks/target-selection.md) and audit/evidence documents ANNOTATED, not revised (SPIKE-connectivity-wireguard-2026-07-03, campaign-10 phaseA-journal) — they record what was true when written and that is their value. Still open and still the operator's, deliberately untouched: target-selection.md's "D-d did not name ep0 either way. Confirm it explicitly."	—
R-91	Old 13 GB datastore copy at /srv/pbs-felhom on ep0's root disk	WATCHING	demo-felhom's first post-migration PBS backup	Delete once it lands; fix CONTEXT.md:1018 same commit	CC
—	First-ever GC on felhom-offsite (armed today 13:11 UTC, never run)	WATCHING	schedule	Sun 2026-08-02 04:30 UTC — confirm it completes	CC
—	demo-felhom's next weekly PBS backup (newest is 2026-07-26)	WATCHING	schedule	~2026-08-02; also releases R-91	CC
—	demo-felhom's next restore-test (84 h cadence, last 2026-07-27 06:38 UTC)	WATCHING	schedule	~2026-07-30 18:38 UTC	CC
R-97	Whole-guest backup tier had no hub signal; quiesce blamed the apps	SHIPPED (controller v0.177.0 + hub v0.78.0/v0.79.0, 2026-07-27)	—	v0.79.0 (R-97c) replaced a FALSE operator-only comment with a real operatorOnlyEvents register	—
F-CRIT-2	A failed offsite backup left a phantom snapshot (1 B, manifest-less, NEWEST) that RESET the tier's freshness clock — 7 days silent on the real 168h cadence, invisible to both the R-88 breaker and the hub deadline monitor	SHIPPED + PROVEN-LIVE (agent v0.106.0, 2026-07-28)	—	NewestArchiveTime now counts only plausibly-complete entries (measured 1 MiB floor; undecidable ⇒ not counted). Campaign fault 2 replayed on demo-hp: phantom rejected + logged once, tier correctly DUE and backed up, and no thrash on the inverse	—
R-99	Server-side prune never removes a phantom snapshot. Confirmed it does NOT count them toward keep-last (dry-run kept 2 real + the phantom) so there is no retention/data-loss bug — but one accumulates per aborted upload, forever	READY (S)	—	Decide a cleanup path. Deletion on a customer datastore is a separate ruling — detection shipped, removal deliberately not automated	CC
F-CRIT-1	An app that fails to restart after a quiesce never alarms on any channel — restartAll discarded the error AND StateStopped was whitelisted on invariant I1, which the quiesce path had made false	SHIPPED + PROVEN-LIVE (controller v0.179.0, 2026-07-28)	—	Both causes fixed. Live on demo-hp: alarmed 9s after grace expiry, banner shows (stopped); a deliberate user stop stayed silent through 9 dead-app scans	—
F-A1	A restore-test in flight made a healthy backup report as FAILED (HTTP 409 read as a tier failure): breaker armed + operator emailed, on both boxes	SHIPPED + PROVEN-LIVE (controller v0.179.0, 2026-07-28)	—	409 → contention: tier stays DUE, dropped before anything stops (15m), and BLOCKED alarm if contention outlives the agent's 120m ceiling (3h). Hub DB: 409 → 0 operator emails, real failure → 1	—
R-100	A restic offsite tier that fails every night never goes stale on the hub — isStale counted from LastRun, which the controller writes unconditionally on failure	SHIPPED + PROVEN-LIVE (controller v0.181.0 + hub v0.80.0, 2026-07-28)	—	Anchored on a new last_success. Severity corrected during Phase 0: this was NOT a silence — backup_failed does fire nightly and reaches the operator (live DB: 5 sends). The real defect is defeated defence in depth: the hub-side pull net was anchored on a field the failing controller keeps refreshing, so it could not compensate for a lost push (cf. F-HUB). Live on demo-hp: induced failure → last_run advanced 11:25:48Z, last_success held 11:24:20Z; demo-felhom healthy → anchor advanced. Legacy degrade logged once per customer, live	—
R-101	Tier-2 LastRun is written on failure and rendered to the customer as „Legutóbbi másolat" — including in the restore confirm dialog	SHIPPED + PROVEN-LIVE (controller v0.182.0, 2026-07-28)	—	CrossDriveBackup.LastSuccess + SuccessTracked; the dialog names the last successful copy and discloses a failed newest attempt. Legacy rows migrate truthfully on first touch (an ok row adopts its time; an error row seeds nothing) — without the marker all 7 fleet rows would have flipped to „Még nincs sikeres másolat" on deploy. Part 2: the three record* sites rebuilt the whole struct; replaced by tier2Update (copy-and-overlay, safe by construction) — the naive fix would have had recordTier2Failure CLEAR the anchor. Live on demo-hp, rendered dialog read in both states	—
C9-F1	Tier-2 „Fájlok visszaállítása" is offered for apps whose copy has no restorable file leg; stops the app, restores 0 files, reports „Nincs hiányzó fájl — minden fájl megvan a helyén."	SHIPPED + PROVEN-LIVE (controller v0.183.0, 2026-07-28)	—	Phase 0 sized it: 43 of 53 catalog apps read NOTHING, 9 read file legs but never their DB/volumes, 1 stateless. Honesty half shipped: Tier2RestoreCoverage refuses UP FRONT without stopping the app and NAMES the working action; a run that proceeds claims only what it examined and discloses that the database and volumes are not covered. Live on demo-felhom: bookstack refused, uptime stayed „Up About an hour" (was „Up 25 seconds"); paperless A1 re-run still byte-identical, 16/16 docs clean	—
C9-F2	An app in a Docker crash loop never alarms on any channel; StateRestarting is in no down-set	SHIPPED + PROVEN-LIVE (controller v0.183.0, 2026-07-28)	—	StateRestarting deliberately NOT added to IsDownState (that alarms on every deploy fleet-wide); a SUSTAINED run becomes down after crashLoopAfter=5m, set above the 120s deploy timeout, Mealie's 60s start_period and R-97b's 180s grace. Dashboard counter uses the same predicate so it no longer contradicts the alarm. Red-proof that matters: the naive IsDownState change fails the brief-restart test	—
C9-F3 → R-104	An interrupted offsite run leaves an exclusive restic lock the self-heal cannot reach: resticStep (offbox.go:634-648) has unlock --remove-all, but ensureOffboxRepo's probe fails first, classifyResticProbe (offbox.go:77-93) has no lock case → "other" → fail-fast. Tier dead until a human unlocks; ClassifyOffsiteFailure likewise has no lock case so the operator is told „A távoli mentés ismeretlen okból nem sikerült" for a precisely-known, self-healable condition	READY (MEDIUM)	—	Add a lock case to both classifiers and let the probe path escalate to unlock --remove-all. Answers Phase C item 8: the repo is NOT usable after a killed run. Cleared manually this run; tier proven working again (ok, 1m35s). Reachable by any interruption — container restart, OOM, host reboot mid-backup	CC
C9-F1b → R-103	Tier-2's restore cannot cover 43 of 53 apps; the action that CAN is the keep-side unit restore (POST /backup/restore → RestoreFromRecoveryUnit, replays volume tars + DB dumps). v0.183.0 NAMES it in the refusal text but does not route to it	READY	—	Put the working action in the card the customer already opened. Deliberately its own task: it places a DESTRUCTIVE operation (overwrites live data with the backup state) behind a button reached via a NON-destructive one, so the confirm copy must carry that difference — the reason it was not folded into v0.183.0	CC
C9-F4 → R-102	Nothing reads the Tier-2 copy's recovery-unit/ mirror. It is written by EVERY Tier-2 run (tier2.go:369, „Unit leg (always)") and read by no code path: RecoveryUnitPath resolves to backups/**primary**/ (appbackup/paths.go:46-48), and the only reader of the secondary tree is tier2_restore.go:79, which reads hdd/+userdata/ only	READY (potentially > C9-F1)	—	Tier-2 exists for the case where the PRIMARY drive is lost — and in exactly that case the primary unit is gone while this mirror survives on the second drive, unreachable by any customer action, leaving offsite as the only route. Verified by enumeration: 6 references to "secondary" in the tree, one writer, one reader, one wipe-warning lister	CC
R-108	Network storage can host an app's namespace, and FileBrowser binds a network share at its ROOT	SHIPPED + PROVEN-LIVE (controller v0.187.0, 2026-07-30)	—	D5 IS UNBLOCKED — that is this item's whole point. RefuseAsAppNamespace (internal/settings/settings.go) is one fail-closed predicate consulted by every placement surface, so no backups/ tree can exist inside the share-root bind. The fix inverted the obvious one: the bind was NOT narrowed, because it cannot be — the :rslave share-ROOT bind is load-bearing (a 2026-07-22 probe proved an in-container access through it wakes the idle automount trigger), there is no userdata/ layer to scope to (apps on a share store at <share>/<app>), and creating one would write Felhom convention onto a customer's own NAS, which R-67 forbids. Operator ruling: refuse the placement, keep the browse bind. FIVE surfaces, not the four this row named — handleStorageDecommission mode=migrate guarded only its SOURCE, so a whole namespace could be decommissioned ONTO a NAS; and the boundary is the deploy POST, not the dropdown (it takes any caller-supplied HDD_PATH, validated only by os.Stat). Nothing stranded: zero apps on network storage across all six customers incl. Peti. R-67's bind verified byte-identical before/after. Supersedes the controller README's "NAS backup locality — decision A". audits/R108-network-app-namespace-2026-07-30.md	—
D5	Move app secrets into the LOCAL recovery unit so Tier-1/Tier-2 restore stop needing the guest	SHIPPED + PROVEN-LIVE (controller v0.188.0, 2026-07-30)	—	CLOSED — the arc's architectural centrepiece is done, and Tier-1/2 no longer depend on the whole-guest tier. A customer now needs the drive and nothing else. Part 0 overturned the brief's own recommendation, on evidence gathered before any code — that is the substantive part of this row. It proposed that only data_key-flagged secrets travel; two findings killed that: (1) the flag is unreliable — only 5 fields across 4 apps carry it, yet n8n/N8N_ENCRYPTION_KEY („Titkosítási kulcs"), wanderer/POCKETBASE_ENCRYPTION_KEY („Adatbázis titkosítási kulcs"), calcom/CALENDSO_ENCRYPTION_KEY and bookstack/APP_KEY carry the SAME labels as flagged adventurelog/SECRET_KEY and are unflagged (→ R-127), so data-keys-only would omit real data keys and the fail-closed gate would not fire for them; (2) a DB password is not resettable in practice — proven on a throwaway postgres:16-alpine: with PGDATA restored from the volume tar, POSTGRES_PASSWORD is ignored (initdb skipped), so a regenerated value fails over the compose network (FATAL: password authentication failed) while the old one still works AND the dump replay still SUCCEEDS via the container's local trust socket — a restore that reports success onto data the app cannot reach. 18 DB/root-password fields affected; MariaDB fails louder (getMariaDBPassword reads the new value against a datadir holding the old hash → Access denied). Operator ruling 2026-07-30: type: secret travels (45 fields), type: password NEVER (7) plus a code register (vaultwarden/ADMIN_TOKEN); plaintext. The exclusion is what LICENSES the plaintext — coupled, not independent. stacks.PortableSecretEnvVars is the single boundary; the register is code, not a catalog flag (a boundary a catalog push can move is not a boundary — R-97a). Precedence: the UNIT WINS over the guest, because the unit's secrets were captured in the same run as the dumps beside them and therefore match the data being restored; pinned both directions. Fail-closed data-key gate UNCHANGED. Manifest → schema 2 + portable_secret_env_vars (names only); schema-1 units still restore from the guest. Live proof on a scratch drill guest through the real endpoints: AdventureLog restored with the guest app.yaml moved aside → secrets recovered=2/2, 27.6 s, then the app read the seeded row over TCP with its own credential (the observable that matters), pre-backup row back / post-backup row gone, no .sql dump so the DB came from the volume tar. Withheld half proven with Grafana: sentinel live in the container, ENC: in the guest, 0 files under the whole backup namespace. 4 red-proofs, each verified to land. audits/D5-drive-alone-restore-2026-07-30.md Flips 07 §3, §7.1, §7.3, §7.4 (new), §8 rows 3/3c/13, §10.1; new capability-map row. Consequence recorded, not changed: the unit already travels to Tier-2 (another customer drive, plaintext, same reasoning) and offsite via restic (encrypted at rest under the customer-owned repo password) — no tier code touched	—
R-127	The catalog's data_key: true flag is UNRELIABLE — at least four data-encrypting keys the catalog itself labels as encryption keys are unflagged; and the O4 restore path can regenerate a DB password that then does not match the restored data directory	READY (S/M)	—	Found by D5's Part 0, and it is why D5's boundary is type: secret rather than data_key. Two separable legs. (a) The misclassification. Only 5 fields across 4 apps set data_key: true (adventurelog/SECRET_KEY, homebox/HBOX_AUTH_API_KEY_PEPPER, papra/AUTH_SECRET, sparkyfitness/{API_ENCRYPTION_KEY,BETTER_AUTH_SECRET}), yet n8n/N8N_ENCRYPTION_KEY („Titkosítási kulcs"), wanderer/POCKETBASE_ENCRYPTION_KEY („Adatbázis titkosítási kulcs"), calcom/CALENDSO_ENCRYPTION_KEY and bookstack/APP_KEY are unflagged — the catalog's own Hungarian labels contradict the flag. D5 makes this non-urgent but not harmless: everything type: secret now travels, so the keys DO reach the drive; what stays wrong is the fail-closed gate, which only refuses for data_key names — so if one of these is missing from both sources the restore proceeds onto data it cannot decrypt instead of refusing. Fix = flag them (app-catalog-felhom.eu, a catalog-only change) + a gate/test that the flag set and the label set agree. (b) The regenerated-DB-password trap. internal/backup/restore_unit.go O4 generates a replacement for any missing non-data-key secret. Proven on postgres:16-alpine: with PGDATA restored from the volume tar, POSTGRES_PASSWORD is ignored (initdb skipped), so the app fails over the compose network while the dump replay still succeeds through the container's local trust socket — success reported, data unreachable. v0.188.0 corrected the WARN's false claim that "stored data is unaffected" and scoped it, but did not add a guard: D5 shrinks this to the rare case (the secret was empty at capture AND absent from the guest). Real fix = either treat a DB password as fail-closed like a data key, or ALTER USER to the regenerated value after the volume restore. 18 DB/root-password fields are in scope; MariaDB fails loudly instead (Access denied), which is the safer half	CC
R-126	A .fab bundle — plaintext secrets, OPTIONAL password — can be exported ONTO a NAS. storageDriveList() (internal/web/handler_export.go) does not filter network paths	READY (S)	—	Split out of R-108, which closed without it: this is an explicit customer-chosen export destination, not a browsing surface reaching a backup tree, so it was never part of D5's precondition (07 §7.3 records that reasoning). Was recorded inside R-108's row as its "second effect, independent of D5"; promoted to its own row so it does not vanish with R-108's closure. Fix = filter network paths out of the export destination list, or require the bundle password when the destination is a share	CC
F-DIAG	Four distinct offsite failure causes collapse into two operator-visible strings	SHIPPED (controller v0.182.0, 2026-07-28)	—	ClassifyOffsiteFailure → quota / orphaned / no_repo / no_units / transport / unknown, each with its own Hungarian message. Unclassifiable says so rather than being folded into a neighbour. Secrets: the old message was a raw err.Error() passthrough carrying sftp:<user>@<host>:<path>; redaction is now by the target's actual host/user/path (a first regex-only attempt leaked on a bare hostname and its own test caught it). Unit-proven; not yet exercised by a live offsite failure of each class	—
F-OPS	A manual pct restore inherits the source guest's bind mounts — during a real DR, on a different host, under pressure	DOCUMENTED (2026-07-28)	—	documentation/runbooks/RUNBOOK-manual-guest-restore.md: which mpN are volumes vs host binds, the mp9 source-VMID trap (it can bind another guest's bootstrap credentials), strip-and-re-add before first boot, and a positive pre-start verification. Docs only by design — the agent already neutralises binds on its own restore paths, and a second implementation would drift	—
F-REBOOT	A guest rebooted during its backup does not come back — shutdown completes, start never happens, no self-heal; 9m47s total appliance outage with every alarm silent	SHIPPED + PROVEN-LIVE (agent v0.107.0, 2026-07-28)	—	60 s guest-power watchdog; onboot is the deliberate-stop discriminator (already the stale-lock path's, and what pve-guests consults), retry bounded 3x/1m-2m-4m then escalates once. Live on demo-hp: 120 s unattended vs the incident's 587 s with a human; Scenario B proven (an onboot:0 guest left stopped)	—
F-LEAK	A failed restore-test cannot destroy its own scratch guest (403 VM.Allocate); the 10-slot VMID band shrinks silently	SHIPPED + PROVEN-LIVE (agent v0.110.0 + host-install v1.21.0, 2026-07-28)	—	Three attempts, two refuted live. (1) Pool adoption: PUT /pools/{pool} also needs VM.Allocate on the VM — membership cannot bootstrap its own authority. (2) Per-path /vms/990000..990009 ACLs: work, but PVE's destroy calls remove_vm_access (LXC.pm:906) which deletes every ACL at /vms/<vmid> — consumed by the op it authorises, one use per slot. (3) SHIPPED: 4th root-fenced exception, band enforced in sudoers literally (pct destroy 99000[0-9] --purge) + in code + at the caller; API destroy still tried first. Live: band PERMITTED, 9201/9100/9999/990010/1 REFUSED, and pct start 990000 REFUSED too	—
F-OBS	deadapp-check leaves NO positive observable on a default (info-level) box — "no alarms" was indistinguishable from "never ran"	SHIPPED + PROVEN-LIVE (controller v0.180.0 + agent v0.109.0, 2026-07-28)	—	INFO summary every 20th scan carrying scans/evaluated/down. Agent v0.109.0 fixes the same shape in the guest-power watchdog shipped hours earlier in v0.107.0 — it logged only at startup and when it acted, so its health could be read only from absence	—
E-2	Drive-role machinery around the moved vzdump target	CLOSED — PARTIALLY PROVEN (Session C, 2026-07-29)	—	CLOSED by audits/SESSION-C-2026-07-29.md. C1/C2 proven in E-2d; C3 and C4 PROVEN LIVE this session (R-114, R-112); C5 FAILED — the gate fires and an alarm reaches the hub, but it is the generic event, not backup_target_absent (→ R-116, the one named open leg). Per the runbook's §9, decided in advance: a failed claim closes E-2 as partially proven with a named leg rather than re-running. The arc's stated definition of done is R-106+R-109, R-108 and D5 — none of which this detour touched. Parts 1–5 complete. Role model + offer-only assignment + Hungarian degraded banner + absent-target signal + installer Case A/B. NOT yet live-proven: the DEGRADED banner and the offer acceptance (both demo boxes are healthy, so neither state occurs naturally) and backup_target_absent end-to-end. Installer is installer-logic-tested, not install-tested — INSTALL-TESTED 2026-07-29 on a fresh nested box via the real ISO/PAIRING route, rc=0 (audits/E2D-fresh-vm-2026-07-29.md §3). Of the "NOT yet live-proven" list: Case B + the degraded state are now PROVEN at the installer and API level; the OFFER ACCEPTANCE is PROVEN at the API level (decline path, restart_required:true, E-2a wrapper, healthy-renders-nothing). Still NOT proven, and now known to be BROKEN rather than merely untested: the banner/offer never reach a customer (R-112) and backup_target_absent cannot fire on device loss (R-113), with the absent-state message itself wrong (R-114)	CC
E-2a	The target move needs a root-fenced wrapper — the agent cannot do it	SHIPPED + PROVEN-LIVE (agent v0.113.0 + host-install v1.22.0, 2026-07-29)	—	felhom-backup-target-apply behind a literal FELHOM_BACKUPTARGET sudoers alias; the agent's PVE role was NOT widened. Enforces F-1 (mountpoint -q) and F-2 (is_mountpoint 1 hardcoded), refuses a root-device target, has NO storage-removal path (grep-assertable), is idempotent and refuses to repoint. All five laws proven live as root on demo-hp with 0 stray storages	—
E-2b	NotifyStorageDisconnected/Reconnected defined and called NOWHERE — a drive going absent emitted no event on any channel	SHIPPED + PROVEN-LIVE (controller v0.184.1 + agent v0.112.0 + hub v0.81.0, 2026-07-29)	—	Seam wired in ReconcileDriveGates; a target drive raises the specific backup_target_absent instead. A keying bug was caught before deploy: a.Path is the registered GUEST path, not the agent's host MountPath, so the target branch was unreachable — every absent drive, target included, fell through to the generic event (v0.184.1). Tests observe the WIRE (httptest hub), not a mock	—
E-2c	E-1 put the whole-guest backups on a drive POST /disks/eject would eject	SHIPPED + PROVEN-LIVE (agent v0.112.0, 2026-07-29)	—	Eject + decommission refuse 409 on the backup-target mount, naming the storage and the remedy. Live on BOTH boxes: demo-hp /mnt/nvme-1tb and demo-felhom /mnt/hdd_1 both refused, drives unmoved. NOT a role reclassification — RoleForStorage untouched, because on both boxes that drive is ALSO the enrolled user-data drive; TestEjectStillAllowedOnANonTargetDrive pins the non-over-correction and /var/lib/vz is still refused by the PRE-EXISTING role gate, not this one	—
PETI	peti-felhom deliberately NOT migrated. Its whole-guest backup still shares a device with its guest, so a drive failure there is offsite-only recovery	ACCEPTED RISK — parked	operator's next visit (tester reinstalling from scratch)	Accepted until the reinstall; re-evaluate if that slips past ~2026-09-01. Do not migrate, do not touch	operator
R-109	The DR recipe records no backup target	SHIPPED + PROVEN-LIVE (agent v0.118.1 + hub v0.83.0, 2026-07-30)	—	backup_target resolves from the PRIMARY tier of cfg.Backup.BackupTiers() — the function the scheduler consults, not a re-derivation — plus the mountpoint, which is what actually separates /mnt/hdd_1 from /var/lib/vz. Three states, and unresolvable is recorded as unresolvable (agent_backup_config_unavailable / not_a_known_storage), never a default. The resolver reads the daemon-start config on purpose: a target move rewrites agent.json and deliberately does NOT restart, so a disk re-read would name a storage no archive had reached. Needed a HUB half nobody had scoped — AssembleDRRecipe allow-lists top-level keys, so the field would have been stored intact and dropped before any operator saw it (→ R-122). Evidence: audits/R106-R109-recipe-completeness-2026-07-30.md	—
R-106	The DR recipe records the PBS namespace as "root" on every box	SHIPPED + PROVEN-LIVE (agent v0.118.1, 2026-07-30)	—	Was open-but-UNREGISTERED on this page until 2026-07-30 (→ R-123) — ROADMAP.md:109 had it READY and the only mention here was inside R-109's prose. Namespace now resolves from the pbs STORAGE (storage.cfg's namespace), the same field vzdump --storage <pbs> makes PVE read, so the recipe cannot disagree with the backup that produced the snapshot. An unconfigured namespace still reads "root" — that is an ANSWER, and namespace_state separates it from not knowing. Live: demo-felhom and demo-hp now report their own namespaces. Evidence: same audit	—
R-122	AssembleDRRecipe silently DROPPED offsite_restic — the offsite recovery location never reached any recipe	SHIPPED (hub v0.83.0, 2026-07-30)	—	Found 2026-07-30 while scoping R-109's hub half; it had already shipped and nobody knew. The controller has emitted offsite_restic since fork-4 ("so DR knows WHERE to recover from"), the hub stored it for all three real customers, and appHalfShape never listed the key — so no delivered recipe has ever contained it. No error, no log, green suite, because the fixture drAppHalf is hand-written and omits the field. hostHalfShape/appHalfShape are ALLOW-LISTS dressed as forward-compat; TestAssembleDRRecipe_CarriesEveryEmittedSection is now the guard, built on halves read verbatim out of the live dr_recipe table. REUSE.md (both repos) records that a recipe section is a TWO-REPO change	—
R-123	R-105 and R-106 were READY in ROADMAP.md with no row on THIS page — each referenced only inside R-109's prose, which is precisely the thread-loss the register exists to prevent	PARTLY CLOSED (2026-07-30)	—	R-106 registered above (and shipped). R-105 still needs a row — it is M-sized, is about three hub-held DR records being {}, and is NOT part of the recipe-completeness set that shipped today. The process gap is the real item: nothing checks that a READY ROADMAP row has an OPEN-ITEMS row. A grep-level gate would catch it	CC
R-124	The recipe spells PBS's root namespace "root", but the PBS API spells it "" and no namespace is literally named root — an operator pasting the field into pct restore --ns root gets a failure	READY (XS)	—	Pre-existing wire convention (ToHub has normalised empty→"root" since slice 6), deliberately NOT changed under R-106 so the field's meaning did not shift mid-fix. Documented at hub.PBSRootNamespace. Affects only a box with no namespace line — no real customer today, all three are per-customer. Fix = emit "" + rely on namespace_state, or emit a --ns-ready form	CC
R-125	A "test through the production path" is only true up to the seam it injects at. agent v0.118.0 shipped an INERT R-106: two tests ran the real Collector.Collect(), but both injected a fakeObserver, and the break was one layer below in mergeConfig — which dropped the pbs namespace, so the recipe still said "root", now with namespace_state: resolved beside it (confident AND wrong, worse than the original defect). Caught by LIVE VALIDATION, not by the green suite	FIXED (agent v0.118.1) — filed for the DOCTRINE point	—	The fix is shipped and red-proofed. What is open is the practice: a production-path claim should NAME the seam it injects at, so the untested span is visible. Candidate for the felhom-testing skill; the state field is also what made the wrong value legible, which argues for the three-state shape generally. audits/R106-R109-recipe-completeness-2026-07-30.md §6	CC
R-89	Retention as a per-customer commercial policy on the hub	READY (increment 2)	—	Policy object + reconciler → ep0 prune job; keep box tokens write-only	CC
R-92	Hub PBS-DR gauge is 0.1 GB-granular — small deltas unverifiable	READY (XS)	—	Widen precision when retention becomes customer-visible	CC
R-93	drill-r50 is both a blocked customer and the only drift fixture	READY (XS)	—	Retire it for a synthetic fixture, or unblock + silence per-customer	CC
R-128	build-felhom-iso.sh:44 comments that ISO_VERSION "aligns with felhom-host-install SCRIPT_VERSION" — a claim nothing evaluated	CLOSED (iso v1.26.0, 2026-07-31)	—	Closed by correcting the claim, NOT by making the assertion real — and that distinction is the durable part. The coupling it asserted does not exist: the ISO is a frozen artifact, while felhom-host-install.sh is fetched at RUN TIME from the website's git-sync of main (→ R-94, R-110), so whatever version an ISO carries, the script a box runs is always current. Making the build read SCRIPT_VERSION and refuse a mismatch — the fix the old row proposed — would have invented a constraint and coupled two things that are deliberately decoupled. build-felhom-iso.sh:45-52 now states the independence in place of the false claim. Do not 'restore' the assertion on the strength of seeing a disabled one	—
R-154	[first-boot] is automated-install-only and nothing in the Felhom tree said so	CLOSED (iso v1.26.0, 2026-07-31)	—	A property of the PVE installer, not a Felhom defect, but one that silently delivered nothing on the path we now ship. Measured with a same-image control in audits/SPIKE-universal-iso-3-2026-07-31.md §2: on an interactive install the hook never runs and the proxmox-first-boot package is not even installed (Config.pm:118 defaults first_boot.enabled=0, set_first_boot_opt is never called, Install.pm:746 returns early, :1360 skips the package, and proxinstall contains zero occurrences of first-boot). Now recorded in pushed source at scripts/iso/pkg/build-deb.sh:6-11 — the header of the mechanism that replaced it — and cited in runbooks/iso-release-gate.md G7	—
R-155	iso-repack.sh refuses any ISO without auto-installer-mode.toml, blocking the no-answer.toml posture	CLOSED (iso v1.26.0, 2026-07-31)	—	NARROWED, not deleted — and the difference matters. The guard protects a real promise: in single-entry appliance mode the menu shows one button labelled "Felhom telepítés" that boots the AUTOMATED installer, and without that file the same button would drop the user into the manual disk-picker. That promise is unchanged and the guard still enforces it for FELHOM_MENU=single (iso-repack.sh:121-128). It simply does not apply to release, where the file's absence is release-gate criterion G1. Do not remove the guard wholesale on the strength of "R-155 closed it" — deleting it would put an unattended installer behind a button promising one	—
R-129	Every doc says demo-hp has "no baked SSH key" and needs the G1 break-glass password — but ssh -o BatchMode=yes demo-hp authenticated by key, first try, 2026-07-31	READY (XS)	—	Stale in the expensive direction: a session that believes it sends itself to the hub vault for a credential it does not need. Verify who owns the key and when it landed, then correct CLAUDE.md, runbooks/target-selection.md:41-42, runbooks/workspace-CLAUDE.md and felhom-agent/CLAUDE.md together — or remove the key if it was not deliberate	CC
R-130	A "hard min" that only warns. A fresh box's local-lvm was ~75 GiB against HARD_MIN_LVM_GIB=120 (scripts/felhom-host-install.sh); the installer logged [WARN] local-lvm free ~75 GiB < hard min 120 GiB and went on to a fully successful install	READY (S)	—	Either the minimum is not hard (rename it and state the real floor) or it is wrong (and 120 GiB is not what a working appliance needs). Leaving it is the R-29 shape: a check that reads as coverage while providing none. Evidence: same audit §8	CC
R-131	sess-f is a fourth orphaned scratch customer on the hub ("R-120 golden 0.186.0 proof", DOWN), left by the 2026-07-30 session	READY (XS)	—	After drill-r50, sess-c, sess-d — the accumulation runbooks/target-selection.md:86-87 and PROMPT-TEMPLATE.md §13 both warn about, now on its fourth instance. Delete it (see the recorded command in audits/tester-gate-golden-0.188.0-2026-07-31.md §7.1); the recurrence itself argues for a periodic scratch-customer sweep rather than another reminder	CC
R-132	curl -w '%{redirect_url}' reconstructs the request URL WITH its basic-auth credential — so a -u ":$HUB_PW" call that never put the password in a URL still printed it	ACTION: rotate HUB_PW	—	Happened on 2026-07-31 while red-proofing the R-120 gate: the hub operator password was written to the session transcript by the write-out format, not by the request. -u is safe; the reporting was not. Rule: read the redirect from -D - and grep ^Location:, never %{redirect_url}, on any authenticated call. Rotate the hub password (/configuration → Login password; ConfigMap auth.password_hash is the reset path) and update ~/.config/credentials	Viktor
R-133	The hub enforces uniqueness on customer_id only — domain is TEXT NOT NULL DEFAULT '' with no UNIQUE/CHECK (hub/internal/store/store.go:114) and the create path only rejects a duplicate id (hub/internal/web/configs.go:644), so two customers can be given the identical domain silently	READY (XS)	—	Harmless while every customer owns their own zone; a real footgun the moment customers share one (the subdomain-onboarding plan). Fix = reject a duplicate domain on create/edit, or warn. Evidence: audits/RECON-subdomain-onboarding-2026-07-31.md §2.2	CC
R-134	Two zone-resolvers disagree on depth. The controller strips labels progressively (controller/internal/cloudflare/zone.go:18); the hub's resolveZone tries the exact name then parentDomain, which strips exactly ONE label (hub/internal/cloudflare/unblock.go:115,136)	READY (XS)	—	For a one-label Felhom-issued subdomain both work; for anything deeper the hub silently fails to find the zone while the controller succeeds — the geo-unblock would then no-op with a "no active zone found" error. One concept, two implementations. Same audit §2.6	CC
R-135	validateCSRF returns TRUE when there is no session cookie (hub/internal/web/server.go:678-683) — measured live: POST with Basic auth and no cookie goes straight past the CSRF gate (404, not 403), while the same POST with a cookie and no token is 403	READY (S) — security	—	Browsers cache HTTP Basic credentials per origin and resend them automatically on cross-origin requests, and SameSite does not govern the Authorization header. So if the operator has ever Basic-authed to the hub in a browser, any attacker page can POST to every mutating route. Latent on the condition, not guaranteed absent. Fix = require the token whenever the request is not provably programmatic, or drop browser-usable Basic auth. Same audit §4.3	CC
R-136	Rename hub_session → __Host-hub_session — makes cookie tossing structurally impossible	READY (XS, one line)	—	Verified on the live production response that all three prefix preconditions already hold: Path=/, Secure, no Domain. Caveat for the ticket: browsers reject a __Host- cookie without Secure, and isSecure is conditional on r.TLS/X-Forwarded-Proto, so plain-HTTP browser access to the hub would stop working (non-browser access uses Basic auth, unaffected). Tested consequence: r.Cookie returns the FIRST match and never tries the others, so a tossed cookie wins outright. Same audit §4.1-4.2	CC
R-137	Cloudflare geo-WAF rules are zone-scoped and non-namespaced — four cross-tenant faults. globalRuleDesc = "[felhom-geo] Global" (waf.go:18) is one literal description per ZONE; appRuleDescPrefix keys by app name with no customer (waf.go:21); BuildGlobalExpression has no positive hostname scoping (waf.go:241); applyDiff deletes every [felhom-geo] rule not in THIS box's desired set (geosync.go:320)	READY (M) — blocks shared-zone onboarding	—	With two customers in one zone: they overwrite each other's Global rule forever; one customer's country policy applies zone-wide; per-app rules collide by name; and disabling the feature for one (or the hub's RemoveGeoRules) wipes them all. Interim mitigation, no code: keep geo-restriction OFF for every shared-zone customer. Fix = namespace descriptions by customer_id + add http.host ends_with "<domain>" to both expressions — a TWO-REPO change (controller + hub RemoveGeoRules). Same audit §5.1	CC
R-138	A shared-zone cf_api_token is a zone-wide DNS-write capability on a customer's box — written 0600 to /opt/docker/stacks/traefik/.env (controller/internal/infra/infra.go:123)	READY (S)	—	Today each box holds a token for a zone nobody else uses, so the blast radius is one customer. Under a shared customer zone, one compromised tester box could repoint every other tester's DNS. The ACME path is already switchable — an empty token selects HTTP-01 (traefik.yml.tmpl) — so the fix is policy plus a guard that refuses to hand a shared-zone customer a zone-scoped token. Same audit §5.2	CC
R-133	The vaulted break-glass console credential is PLAINTEXT AT REST — every hub DB backup is a fleet-wide console-credential dump. host_recovery.secret holds each managed box's root@pam password verbatim, so any copy of the SQLite DB (Longhorn snapshot, PBS backup of the hub PVC, a hand-taken copy during a diagnosis) carries root console access to every Felhom host in one file	READY (M) — NEW 2026-07-31	—	The deferred leg of hub v0.84.0 (Console access card), filed separately because v0.84.0 changed only WHO can retrieve the secret, never how it is stored. v0.84.0 makes it more worth doing, not more broken: retrieval now rides the hub SESSION, so the DB and the login password are jointly the whole protection (ruling S-4, CONTEXT.md). Fix shape: envelope-encrypt the host_recovery.secret column under a KEK held outside the DB — the hub already proves it can hold something it cannot itself read (escrow blobs), and that contrast is the argument. Two constraints the design must respect: the credential must stay retrievable when the box is unreachable (that is the whole point of break-glass), so the KEK cannot live on the box or depend on the agent; and the global-key API path must keep working with the hub UI down. Would flip the capability-map row "Break-glass management-plane recovery", which today reads IMPLEMENTED with this as its caveat	CC
R-156	An app's data is neither persisted nor backed up, and it reports healthy. A template mounts a volume at a path the application never writes, so the data sits in the container's writable layer: lost on redeploy, and tarred nightly as an empty directory while the healthcheck stays green. papra (Campaign 10) and gramps-web + wishlist (the 53-template sweep) all convicted.	CLOSED — all three apps fixed (papra template, 2026-08-03)	—	The gate SHIPPED: app-catalog-felhom.eu/scripts/check-volume-persistence.py (runtime probe; docker diff + mount-occupancy + writability, canary self-test, fails closed). It convicts papra /app/data[vol,EMPTY] → db.sqlite in the writable layer. gramps-web and wishlist were FIXED in the sweep; papra was NOT — it is referred, because the fix needs either the app to use /app/data or the template to mount /app/app-data. THE REFERRAL IS RESOLVED, 2026-08-02 — papra is deployed NOWHERE, so the template fix strands nothing and can be applied. The referral existed because changing where the volume mounts moves live data: an installed papra writes db.sqlite into the container's writable layer, and a remount relocates the path out from under it. With no instance deployed there is no live data to move, so the cheaper leg — the template mounts /app/app-data — is takeable directly, without waiting on upstream to adopt /app/data. Provenance, stated because it decides the row: the observation is docker ps -a on demo-hp's guest 9201 returning empty, supplied with the 2026-08-02 task; this session did not re-measure (documentation-only, every box fenced). Scope of that evidence, honestly: it covers guest 9201 — the guest papra was convicted on in Campaign 10 — and no other customer's guest was enumerated, so a re-check belongs in the task that edits the template, before it edits it. Next action: apply the template fix in app-catalog-felhom.eu and re-run scripts/catalog_gates.py (deliberately not done here — that repo was out of scope for this task). See R-161 (nothing runs the gate automatically) and R-159/R-160. CLOSED 2026-08-03 — papra fixed, and the precondition was CHECKED rather than inherited. The 2026-08-02 evidence covered only guest 9201 and both demo boxes have been wiped since, so it was re-measured three ways: docker ps -a (INCLUDING stopped containers) on both demo guests → no papra; and the hub's /hosts fleet view → exactly two enrolled hosts (demo-felhom-8363b5, demo-hp-bb76ea), zero papra references. Deployed nowhere ⇒ the template fix strands nothing. The fix decided from the IMAGE, not the README or the upstream docs: docker inspect of ghcr.io/papra-hq/papra:26.6.1-rootless gives WORKDIR=/app and all three data paths under ./app-data — DATABASE_URL=file:./app-data/db/db.sqlite, DOCUMENT_STORAGE_FILESYSTEM_ROOT=./app-data/documents, PAPRA_CONFIG_DIR=./app-data — and /app/data does not exist in the image at all, so the old mount pointed at a path nothing could ever write. Departure from the task's stated preference order, recorded because it was deliberate. Option (1) — reconfigure the app to write where the template already mounted — WAS available (all three paths are env-settable). It was not taken: it enumerates data paths, so a fourth one added upstream would silently escape to the writable layer again, which is R-156's exact failure mode re-armed and invisible. Mounting the app's own data ROOT (papra_data:/app/app-data) captures every current AND future path by construction. One line instead of three env vars with an ongoing coupling to upstream. The runtime gate is the arbiter and it was run, in BOTH directions. check-volume-persistence.py papra → CLEAN, and its mandatory self-test passed on that run ('prober flags the R-156 signature and clears a correct template — trustworthy'), so the verdict carries a live proof that the instrument discriminates. Red-proof on the real template, not just the canary: reverting the mount to /app/data and re-running → BROKEN, with the exact R-156 evidence — 'mount /app/data is NOT writable by the app's own uid=999', 'DATA in the writable layer at /app/app-data/db (db_signature=True, e.g. [db.sqlite])', 'declared volume /app/data is EMPTY'. Restored, re-run, CLEAN. Two operational notes for the next person to run this gate: it needs root (it reads volume contents under /var/lib/docker/volumes, mode drwx--x---; as a normal user its own canary self-test fails UNDETERMINED and it correctly refuses to report), and it hardcodes a scratch path /srv/felhom-gate. Run it scoped to the app you touched — unscoped it deploys all 53 templates and takes far longer than a session allows	—
R-157	bootrecon's start-ONCE sweep misses the boot orphan it exists to recover — TWO mechanisms.	CLOSED — SHIPPED + PROVEN-LIVE (B: controller v0.189.0; A: v0.190.0, 2026-08-02)	—	Both mechanisms closed. (B) the container-count signal → recorded intent (R-166). (A) the sweep looked ONCE at T+5 s, deriving candidates from a fleet docker was still restoring — 3 of 6 hard resets. Now a settle-then-sweep window: sample the fleet every 5 s, settled after 3 identical samples, sweep ONCE at the end; ends on settled OR a 50 s budget, and the log says which. The budget is 50 s because a test rejected 60 s: settle+budget+one 30 s retry must stay under the 90 s deadAppBootGrace or a successful recovery stops being silent; 60 s gave 95 s. A window that genuinely overruns emits a LATE RECOVERY WARN naming the apps — the grace was NOT widened to hide it (§8.3). A defect in the fix, found by live validation not review: GetStacks() is the Manager's cache, refreshed by the scheduler every 10 s, so sampling it every 5 s without refreshing let "settled" mean "the cache did not update" — observed missing a container removed 5 s before the window closed. sampleBootFleet now refreshes first. Live: 6/6 hard resets on the shipped build, every app back every time (settle times 10/40/10/10/15/15 s — i.e. the window routinely waited 2–8× longer than the old fixed 5 s), plus a before/after on ONE app on ONE box: the pre-fix window logged no boot-orphaned apps for calibre-web at 18:08:35, the fixed one found and recovered it at 18:18:50	—
R-170	The drive-backed boot gate infers a customer's Stop from a container count.	CLOSED — SHIPPED + PROVEN-LIVE (controller v0.190.0, 2026-08-02)	—	shouldRecreateOnBoot now reads desired_state with the SAME three-way table as isBootOrphan: stopped → never; running → recreate whatever the container count; absent → exactly the pre-v0.190.0 hasContainers behaviour. presentStable untouched and still load-bearing (an absent drive is never recreated here — the very term the boot sweep was missing, R-171). Its comment argued at length FOR the container count and was rewritten; a correct implementation under a comment arguing the opposite is worse than either alone. The agreement is pinned from BOTH sides against one fixture table (TestBothBootGatesAgreeOnIntent / TestShouldRecreateOnBoot_AgreesWithBootrecon) because the two gates cannot be called from one package without an import cycle. Live on 9201, both halves in one reboot: calibre-web (drive-backed, running, ZERO containers) → recreating drive-backed app calibre-web; immich (stopped) → 1 drive-backed app(s) left stopped on purpose	—
R-171	The boot sweep started apps whose data drive was ABSENT — a regression introduced by v0.189.0, now FIXED. Replacing isBootOrphan's container-count term with recorded intent made a drive-gate-stopped app (compose down ⇒ zero containers, and the gate never touches desired_state because it is not the customer) read as a boot orphan	CLOSED — SHIPPED + PROVEN-LIVE (controller v0.190.0, 2026-08-02)	—	Reasoned from the diff, then CONFIRMED on hardware before any fix was written (audits/DIAG-bootrecon-drive-absent-2026-08-02.md). The sweep found and started calibre-web with its drive unmounted, burned both attempts and handed it to the dead-app alarm — a false alarm about an app the drive gate is deliberately holding. The write hazard did NOT materialise: compose failed mkdir …/userdata: permission denied because the unbound mountpoint is host-root-owned and the guest is unprivileged — an accidental protection no code owns, no test pins, and one chown or one privileged guest away from gone. Fix: new consumer-side seam bootrecon.StartGate, fail-safe (cannot determine ⇒ do not start), wired in main.go; Manager.DriveLive reuses the userdata belt's own isMountPoint seam so the two cannot drift. The rule is not new — the API's startGatedByMissingDrive already refused this to the customer; the sweep bypassed it by calling Manager.StartStack directly. Widening the window (R-157 A) made two more holders reachable, so the same seam also refuses an app held by a quiesce or an in-flight app-data operation (§8.2), reusing quiesce.SuppressedStacks() and a new read-only AppStopGuard.HeldStacks(). Held apps report as HeldByDrive, never StillDown — that is the alarm's bucket. ID established free: grep -ro "R-171\b" documentation/ *.md → 0 hits before minting	—
R-172	A false host_stale alarm fires when the hub's SQLite refuses two consecutive host reports.	CLOSED — SHIPPED + PROVEN-LIVE (hub v0.88.0, 2026-08-02)	—	ROOT CAUSE WAS NOT TUNING — THE PRAGMAS WERE NEVER APPLIED. store.New used ?_journal_mode=WAL&_busy_timeout=5000, which is mattn/go-sqlite3 syntax; the driver is modernc.org/sqlite, whose applyQueryParams reads only _pragma/_time_format/_time_integer_format/_txlock/_inttotime and ignores the rest without an error. The hub ran in rollback-journal mode with busy_timeout=0 for its entire life while its own source said WAL — a configuration asserting an invariant the code did not provide. Proof: a 128 MB open /data/hub.db with no -wal/-shm beside it. Fix: ?_pragma=journal_mode(WAL)&_pragma=busy_timeout(5000)&_txlock=immediate. _txlock=immediate is not optional — database/sql's Begin() is DEFERRED, so a read-then-write tx must upgrade its lock and a failed upgrade is SQLITE_BUSY_SNAPSHOT, which busy_timeout does not retry; this store has 10+ db.Begin() sites, all write paths. Retry options (b) and (c) were deliberately NOT taken — with readers no longer blocking writers a surviving SQLITE_BUSY would be a real signal, and a retry would hide it; revisit only on evidence. Live: -wal+-shm now present, zero SQLITE_BUSY since rollout, host back to ok, and PRAGMA integrity_check = ok with journal_mode=wal after three unrelated OOM restarts. Operational consequence handled: a WAL DB cannot be copied by taking hub.db alone — the break-glass retrieval in operations/nodes.md did exactly that and is now WAL-aware (the live -wal was 729 KB, i.e. a bare cat would have silently omitted it)	—
R-174	The app-stop guard's crash recovery started apps onto MISSING drives — a regression in v0.189.0 code.	CLOSED — SHIPPED + PROVEN-LIVE (controller v0.191.0, 2026-08-02)	—	Found by REVIEW on 2026-08-02, in code shipped 2026-08-01, and closed the same session — R-171 one path over. appStopGuard.SetStarter(stackMgr) handed Recover the RAW stack manager, whose StartStack has no drive gate, and Recover runs at startup — exactly when an external drive may not have come back. So: a backup stops an app, the box loses power, the drive does not remount, and the app is started on a missing drive. The rule was not new — the API's own startGatedByMissingDrive already refused this to the customer; the guard bypassed it. bootDriveGate could NOT be reused whole, and the reason is recorded in the code: its holder #2 reads bootAppStopGuard.HeldStacks(), which during Recover is the guard's own marker — it would refuse every recovery it was meant to perform — and holders #1/#2 read package-level vars assigned AFTER Recover() runs, so a whole-gate reuse would be correct only by accident of nil-safety. Holder #3 is extracted into a shared driveStartGate with two callers, one implementation, and TestBootDriveGateAndAppStopShareTheDrivePredicate pins the delegation. A REFUSAL IS NOT A FAILURE: new ErrStartRefused + a Refused bucket — both keep the marker, only Failed alarms, because routing a deliberate hold into NotifyBackupFailed (customer-enabled by default) is the very R-171 false alarm this fixes. main.go guards on Alarming(), not != nil, and the pre-existing seam test was TIGHTENED to require it. Live on 9201, both directions: drive held unmounted → refusing to restart "calibre-web" … drive /mnt/felhom-drives/hdd_1 is not a live mountpoint, marker retained byte-identical, zero containers started, not alarming; drive returned → restarted calibre-web, marker CLEARED. ID established free: grep -ro "R-174\b" documentation/ *.md → 0 hits	—
R-175	07-backup-architecture.md §7.5 states ONE box's size bound as if it were the fleet's.	CLOSED — FIXED 2026-08-03 (same pass as R-165)	—	Measured, not inferred (audits/SPIKE-r165-mp1-merge-2026-08-02.md M1: pct config 9201 on both hosts). Independent of the merge — the sentence is wrong today and will be wrong differently after R-165. The fix is to state the bound as a FUNCTION of mp1, not a constant, and to say which box any quoted figure came from. Same class as the comment-asserting-an-invariant rule: a doc stating a fleet-wide number that only one machine satisfies reads as settled and is not. ID established free: grep -ro "R-175\b" documentation/ *.md → 0 hits FIXED. §7.5 gained a 7.5.1 which (a) states plainly that the bound is a FUNCTION of mp1 and applies only to a box still on the split layout, naming all three real shapes, and (b) records that the ceiling itself has been removed by R-165 for boxes built from golden ≥ 0.192.0. Fixed in the same pass as the merge rather than filed and forgotten, because the section would otherwise have been wrong in two ways at once	CC
R-176	Two prerequisites for the R-165 merge are UNMEASURED, and both are cheap. (a) Whether a pre-merge archive (carrying mp1) restore-tests cleanly into a merged-layout guest — reading mountParity (felhom-agent/internal/reconcile/restoretest.go:347) says it should, because the restore recreates mp1 from the archive so archive and restored guest agree; that was reasoned from source and never executed. (b) The in-place per-box migration (move <mp1>/felhom-data onto mp0, drop the slot, verify) has never been rehearsed even once, so "is the box restorable at every point of it?" is currently unknown	(a) ANSWERED 2026-08-03 (P1: PASS). (b) NOT REQUIRED — operator ruling: every node is reinstalled, none migrated	blocks R-165 landing safely	Filed because this project's own record is that FOUR production designs specced against unvalidated mechanisms were all wrong — which is exactly why R-165's own spike refused to design. Both are one command on a Tier-0 box (D-d: both demo boxes are disposable). (b) is only required work if Peti's box turns out to need migrating rather than reinstalling — the hub cannot answer that (M5: peti-felhom exists as a customer with no host in the register), so it is the operator's input. ID established free: grep -ro "R-176\b" documentation/ *.md → 0 hits UPDATE 2026-08-03. (a) is measured and passed — audits/SPIKE-r165-phase0-2026-08-03.md P1: a real pre-merge archive (mp0+mp1, confirmed from its own vzdump log) restore-tested on demo-hp, pass: true, mount_parity: ok, 84 s, with mountParity untouched. One limit stated rather than glossed: it ran with the pre-merge agent because the merged one did not exist yet, and the comparison is archive-vs-its-own-restore which never consults the host layout — re-run it once against agent v0.120.0, which is one command. (b) is withdrawn, not deferred: the operator ruled that every node is REINSTALLED rather than migrated in place (both demo boxes are Tier 0; the colleague's box carries none of our customer data and is clean-installed in a few weeks), so the in-place migration rehearsal has no consumer. Recorded explicitly rather than silently skipped	CC
R-183	A fresh install fetched the vouched agent BINARY and its sixteen CONFIG files from two different refs, and nothing compared them. felhom-host-install.sh resolved the agent version from the hub manifest and sha-verified the binary — then took felhom-agent.service, felhom-agent.sudoers and fourteen more from raw/branch/main, i.e. whatever the agent repo's tip happened to hold at that second. One install, two refs, no comparison	CLOSED — SHIPPED (installer v1.23.0, 2026-08-03)	—	Found while implementing R-110, by reading fetch_raw's call sites rather than the spec's description of them — the task said nine files from felhom.eu; they are sixteen and they come from felhom-agent. Why it is a defect and not only untidiness: these files are the agent's own operating surface — its systemd unit, its sudoers, its guarded wrappers — and configs/felhom-backup-target-apply is installed 0755 into /usr/local/sbin and root-fenced in sudoers, validated only by bash -n. A config newer than the binary is a root-executed artifact the vouched version was never tested against. Not hypothetical in shape: the fleet has shipped exactly this class before, where a config and the code that reads it moved independently. Fixed by pinning to the agent version the install is already committed to, on the operator's ruling: raw/tag/v$ART_AGENT_VER/, resolved from the hub manifest that the binary's sha is already checked against — so binary and configs now come from ONE ref by construction. No fallback to a branch, deliberately: a missing tag dies loudly rather than quietly serving main. Pinned by hostinstall_gates.py gate 6 (no raw/branch/ anywhere; the $ART_AGENT_VER pin still present), red-proofed by reverting one of the sixteen and by removing both assertions. felhom-agent now carries v<semver> tags (v0.120.0 created retroactively at cd6e267, the commit the published binary was built from; configs/ is byte-identical there and at main, so nothing depended on the choice) and release-agent.sh creates them as part of releasing	—
R-184	Nothing prevents the hub from vouching an agent version that was never released. The R-115 gate proves every RELEASED version is installable, but it works from git tags — so a hub artifact-manifest entry naming a version with no tag and no package is invisible to it. The installer would then die at step 5 on a virgin machine, as root	READY (S) — NEW 2026-08-03	—	Filed BECAUSE the R-115 gate deliberately does not cover it, rather than leaving the gap unstated. CI cannot check it: the hub's /api/v1/artifacts/<customer> answers 401 without a per-customer retrieval passphrase and Gitea's package listing api answers 401 without a token (both measured 2026-08-03, P-C), so a credential-free gate can ask "is this version installable" but never "which version is vouched". Two shapes, and the second is better: (a) give CI a hub credential — expands what CI can reach, and is the operator's call not a gate author's; (b) validate at vouch time, in the hub: the operator UI's Day-0 artifact form refuses a version whose package is not downloadable. (b) fails closed at the moment of the decision, needs no new credential anywhere, and puts the check where the mistake is actually made. Exposure is low and should be said so: vouching is a deliberate operator action against a version they have just released, and R-115's release path now makes released-but-unpublished nearly impossible. This is the residue, not the main risk	CC
R-182	A full disk tells the operator about ONE app and silently swallows every other app's refusal for an hour. The hub's operator cooldown key is customerID + ":" + eventType + cooldownTierSuffix(details) (hub/internal/notify/dispatcher.go:268). recovery_unit_capture_failed carries app in its details and no tier, so the suffix is empty and the key contains no app identifier: the first refused app's alert takes the 1-hour slot and the second app's is dropped — and dropped BEFORE LogNotification, so it leaves no row on any channel. It cannot even be audited after the fact	CLOSED — SHIPPED (controller v0.194.0 + hub v0.90.0/.1, 2026-08-03)	—	FILED AS THE OPPOSITE DEFECT AND THE MEASUREMENT OVERTURNED IT. It was filed 2026-08-03 as "the reserve re-alerts on every status refresh" — too MANY alerts — from the controller-side observation that a second push followed 13 s after the first. That was the sending end. Measured at the receiving end (hub notification_log + events, read from a copy taken WITH its -wal, freshness confirmed by the newest row post-dating the session): 9 events received today → 2 operator e-mails sent. 06:40:03 privatebin AND opengist both refused → opengist e-mailed, privatebin's alert has no row at all. 08:59:46/47 opengist AND privatebin both refused → privatebin e-mailed, opengist's absent. 08:59:59, 09:03:00, 09:07:06 → no operator row whatsoever, all inside the 1-hour cooldown opened at 08:59:47. So the controller pushing repeatedly is not the defect; the hub emitting at most one operator e-mail per customer per hour is, and the loser is silent. This is R-97a's failure mode exactly, in a second event type. That row's own comment states it: "felhom-pbs failing at 09:00 would swallow local failing at 09:20 for the whole hour". cooldownTierSuffix was written NARROW on purpose — empty unless the producer sends a tier — so no existing type's behaviour changed; recovery_unit_capture_failed simply never opted in. CORRECTION OWED, and it is the reason this was worth measuring: the 2026-08-03 R-181 report said "one recovery_unit_capture_failed per app, HTTP 200". That was true of what the CONTROLLER pushed and would be read as the operator was told about each app — which is false. The distinction between an accepted event and a sent e-mail is the whole of this row. Fix shape (NOT implemented — Part 0 was investigation only, by instruction): let the producer opt into a per-app cooldown key, the way R-97a let the whole-guest producer opt into a per-tier one — the narrow mechanism already exists and needs no widening. And a suppressed operator alert should leave a skipped row rather than nothing, or this class stays undiagnosable from the hub's own records. CLOSED 2026-08-03 — controller v0.194.0 (88897a2) + hub v0.90.0/.1 (dd40f85, f21e7ca), proven live on demo-hp. The operator ruled against the obvious fix (putting app in the cooldown key): on a full disk that is one e-mail per app, the volume problem wearing the correctness problem's clothes. What shipped instead is one digest per run plus a record for every failure. (1) The silent drop is gone. processOperator's cooldown no longer returns bare — a suppressed operator event writes a suppressed row carrying the message and the key that suppressed it, for every operator event type, not only the one that exposed this. The cooldown's duration and semantics are deliberately unchanged: this makes the drop visible, not absent. (2) backup_run_failures — one operator mail at the end of a run listing every failed app, its leg and its reason, with the counts and the free space; in allowedEventTypes AND operatorOnlyEvents, with a test demonstrating a customer who has enabled it receiving nothing. (3) recordOnlyEvents — a third routing class. recovery_unit_capture_failed is now stored + logged every time and never mailed: the record and the notification are different things, which is the durable half. A register rather than a severity downgrade, which would have relabelled a genuine failure as informational everywhere it is queried. (4) cooldownRunSuffix — a sibling of cooldownTierSuffix, so tier keeps byte-identical semantics and R-97a is untouched. A unique run_id per real run means the cooldown can never collapse a manual run into the nightly one (the operator ruled on that explicitly). A GAP THE SPEC DID NOT ANTICIPATE, AND ITS FIX. recovery_unit_capture_failed also fires from GetFullStatus's periodic sweep, outside any run. Making it record-only would have left those failures recorded and never notified — a NEW silence created while closing one. So the sweep emits a digest too, deliberately with no run_id, so it stays under the ordinary hourly cooldown exactly as before while the mail now lists every failing app instead of whichever was first. A refusal is noted ONCE, where the verdict is taken, not at the three legs that consult it — R-181's contract is one verdict per app per run, and noting it per leg listed a single refused app three times and produced counts like "2 of 1 apps failed". Found by the digest's own test, not in review. LIVE PROOF, demo-hp, the same method that produced the original measurement — a real 64.6 GiB fallocate to 241 MB free (thin pool held at 30.78 → 30.78, instrument re-proven). The hub's notification_log for the run reads: `recovery_unit_capture_failed	operator
R-181	The capture floor guards the cheap leg and not the leg that fills the volume — and its refusal message asserts an invariant the code does not provide. B2 (controller v0.192.0) is recorded on R-165 as the deliberate replacement for the bulkhead the mp1 partition used to give. It is consulted in exactly one place — m.unitFloorBlocked(stack.Name) at recovery_unit.go:328, inside captureAllRecoveryUnits, which writes a manifest and a compose copy: a few KB. The leg that writes the bulk, runVolumeDumps (backup.go:535), has no floor check at all — its gates are protected-stack, volume-less, disconnected, decommissioned — and it runs FIRST, by design ("MUST run before captureAllRecoveryUnits so the manifests enumerate the fresh tars", backup.go:483). So the write that fills the filesystem is unguarded, and the floor then refuses the write that would have cost almost nothing. Second limb: the refusal message is false. recovery_unit.go:331 prints "the previous unit is untouched and NOTHING was deleted". Nothing was deleted — true. Untouched — measured false: privatebin's volume-dumps/privatebin_privatebin_data.tar went 26c546c2… → b538ab89… and opengist's went 182,272 B → 2,147,666,432 B, both rewritten by the earlier leg, while each unit's manifest.json kept created_at: 2026-08-03T06:34:26Z and its checksums block covers only the three compose files — so a unit's payload can be swapped under a stale descriptor and nothing in the unit can detect it	CLOSED — SHIPPED (controller v0.193.0 + v0.193.1, 2026-08-03)	unblocks R-165	FIRST LIVE FIRING OF B2, and it is why the runbook asked for one. Proven on demo-hp 2026-08-03 06:40:03 on a box reinstalled from the merged golden (R-178). Method: a real 2 GiB file in opengist's data volume, then fallocate to bring the filesystem to 96 % used / 3.0 GiB free — both floor terms deliberately still clear, so the run started. The fallocate instrument was proven before use (5 GiB moved guest df 977M→6.0G while thin-pool data_percent stayed 29.03 → 29.03: zero blocks allocated), because demo-hp's thin pool is 53.93 GiB and a real fill to 97 % of a 70 G volume would have exhausted it and corrupted every guest on the box including the drill-r50 fixture. Sequence observed: opengist's volume dump wrote 2.0 GB unguarded → free fell to 1.0 GB → both apps' recovery-unit captures were then REFUSED on the 1.0 GiB free term, each pushing recovery_unit_capture_failed (severity error) to the hub, accepted HTTP 200. What DOES hold: it refuses per app rather than aborting the run, it never deletes, and the alert reaches the operator. Fix shape, not written this session by design (§7 of the runbook): the floor belongs before the write in runVolumeDumps too, the message must stop claiming what the earlier leg has already falsified, and per CLAUDE.md "a comment asserting an invariant needs a test pinning it" the pinning test must assert the consequence (after a refusal, is the previous unit's payload byte-identical?) and not the mechanism. Class: the sixth entry in CLAUDE.md's own table of shipped guarantees the code did not provide — found, as four of those were, only on live hardware. CLOSED 2026-08-03 — controller v0.193.0 (fef07c3) + v0.193.1 (6c43bf6), proven live on demo-hp. The fix is ONE admission verdict per app per run (internal/backup/admission.go), taken before that app's FIRST write and consulted by all three legs — the three write under one per-app root (appbackup.RecoveryUnitPath), which is exactly why one verdict can honestly cover them. Decided LAZILY at the app's first write, never once at run start: app A's dump can put app B under the reserve, so a run-start verdict would read a disk that no longer exists — the same class of mistake one level up. Never re-decided between an app's own legs (that IS the split this closes) and reset per run. Placed ahead of DumpAppVolumesSafe, which stops the stack as its first act, so a refused app is never bounced; placed AFTER the volume-less check, which has no write to gate. Exactly ONE operator alert per refused app per run. Leg order unchanged. The floor is now SIZE-AWARE, which is the second half of the defect: it asks whether THIS app's write would cross the reserve, not only whether the filesystem is already below it — the term whose absence admitted an app at 96% and then let it write 2 GB. Estimate = the app's previous .sql+.tar on disk; no history → headroom-only deliberately, or the first backup becomes the one that can never happen, and the alert says so. A container-based du was MEASURED and rejected, not assumed: 66 timed runs on demo-hp guest 9201, median ~355 ms/volume (341–404) on volumes holding tens of KB — the cost is container start-up, not the walk. Decisive on top: docker run needs the writable layer, so the instrument can fail under exactly the pressure the reserve exists to handle; and the previous-dump estimate measures the ARTIFACT that will be written rather than the live volume. THE MESSAGE WAS NOT WEAKENED — the behaviour moved so the wording became true, and it is checked by sha256 tree fingerprint, not by reading the log line (which is what lied). LIVE PROOF, demo-hp guest 9201, the same method that found it. The fallocate instrument was RE-PROVEN on the rebuilt box before use (guest df 1.2G→6.2G on a 5 GiB step while thin-pool data_percent stayed 36.83 → 36.83: zero blocks allocated), because a real fill of a 70 G volume would exhaust the 53.93 GiB pool. Headroom term @ 08:59:46 — 906 MB free / 99%: both apps refused, TREE_SHA 111d1760c18d3440f700634ab325f8b8 IDENTICAL before and after (10 files, incl. opengist's tar still at 182,272 B — R-181's own 'before' figure), no Stopping <app> for safe volume dump line at all (it is present in the 08:58 baseline, which is what makes its absence evidence), 0 volume dumps, one recovery_unit_capture_failed per app HTTP 200. Freed and re-run @ 09:01:33 — both captured normally. SIZE term proven separately @ 09:03:00, reproducing the original sequence with a real 2 GiB file in opengist's volume (its previous tar then 2,147,666,432 B, the exact live figure) and the filesystem at 91% used / 2.9 GB free — both headroom terms deliberately clear: opengist refused (size) — "this app's last backup was 2.0 GB and writing it again would cross the reserve" — while privatebin was ADMITTED and dumped normally, proving the term is per-app and not a global halt. Teardown complete: fill removed, planted file removed, pct fstrim 9201 returned 67.5 GiB, pool 29.43% (below the 36.83% baseline), tree byte-identical to the pre-test fingerprint. v0.193.1 shipped in the same session, found by this very proof run: the estimate was rendered fixed to 2-decimal GiB, so opengist's real 178 KB printed as estimated 0.00 GiB write — which reads as no estimate was available and is the opposite of what happened. Rendering moved to humanizeBytes; arithmetic unchanged. Re-verified live: estimated 178.0 KB write. 11 new tests + 4 red-proofs, each demonstrated failing then restored: both dump-leg gates removed (= v0.192.0) → Scenario A red with the tree shown changing; the size term removed → Scenario D red; a prune injected into the refusal path → Scenario F red; the floor moved above the warning band → Scenario G red. Recorded honestly: the specified Scenario-F mutation (remove the reserve entirely) did NOT turn F red — removing it makes every app write, which overwrites and adds but deletes nothing, so a deletion-watching test correctly stays green; the prune mutation is the one that proves the assertion. The DB leg cannot run without Docker, so its gate is pinned by an AST walk of backup.go asserting admitApp precedes DumpOne — strings.Contains is insufficient, a commented-out call still contains the string. §3's correction CONFIRMED in passing and not chased: restore_points.go:57-59 takes the manifest mtime then newestArtifact over .sql and .tar, so the newest of the three wins — the restore point does NOT show a stale timestamp. New finding from the live run → R-182.	—
R-180	--archive-storage is accepted without checking the agent's token will ever be granted on it, and the failure lands at step 8/8 — after the root@pam password has already been rotated. felhom-host-install.sh validates the archive storage EXISTS (pvesm status --storage, :1583) and that the golden volid RESOLVES on it (:1661), both in pre-flight. It never checks that storage against the ACL set it is about to grant, which is the fixed default local local-lvm felhom-pbs (--acl-storages, which runbooks/day0-install.md tells the operator not to pass). A storage outside that set therefore passes every pre-flight gate and dies at the last step	READY (S) — NEW 2026-08-03	—	Hit live on demo-hp 2026-08-03 during R-178 Phase A, self-inflicted and therefore a clean demonstration: the golden was staged on felhom-backup (the enrolled NVMe, where the box's vzdumps live) and --archive-storage felhom-backup passed. Pre-flight passed; steps 1–7 ran; step 8 returned reconcile: bring-up restore: proxmox: POST /nodes/felhom-host/lxc -> HTTP 403: permission denied at /storage/felhom-backup (missing privilege Datastore.AllocateSpace). The cost is the ORDER, not the error — by the time it fires, step 2 has minted the PVE token, step 4b has rotated root@pam and vaulted it (so the old console password is already dead), and step 5 has installed the agent. Recovery was --resume after moving the golden to local, which worked cleanly. This is statically checkable in pre-flight: ARCHIVE_STORAGE ∈ PVE_STORAGES is a one-line assertion over two variables both known at :1583. Same class as R-29 — the checkable thing that nothing checks	CC
R-179	--uninstall leaves the NAS network-storage systemd units behind, with the automount in failed state and the parent bind still mounted. The teardown's residue-diff provenance (day0-install.md Part E: "a full-filesystem diff against the pre-install baseline showed zero Felhom-named leftovers") is from v1.9.1, which predates the NAS network-storage feature. A box that has ever had a network share configured keeps /etc/systemd/system/mnt-felhom\x2ddrives-<share>.mount and .automount after a full uninstall	READY (S) — NEW 2026-08-03	—	Observed on demo-hp 2026-08-03 after --uninstall --vmid 9201: mnt-felhom\x2ddrives-Felhom\x2dShare.automount loaded failed failed, its .mount loaded inactive dead, and mnt-felhom\x2ddrives.mount still active mounted — the uninstall's own output had warned /mnt/felhom-drives/Felhom-Share is busy — NOT forcing and /mnt/felhom-drives root bind left mounted, which is correct behaviour (it never forces an unmount) but is not teardown. Cleared by hand before the reinstall: stop both units, remove both unit files, daemon-reload, unmount the autofs then the parent. NEGATIVE CONTROL, same day: demo-felhom's uninstall left nothing (`ls /etc/systemd/system	grep -i felhom→ only the unrelatedfelhom-bootstrap.service; no felhom mounts) — because that box had no network share configured. **So the residue is conditional on the feature having been used, which is exactly why a diff taken on a box that never used it reported clean.** felhom-bootstrap.serviceis NOT residue — it is the ISO first-boot unit,disabled+inactive`, exactly-once and already fired
R-178	The merged golden (0.192.0) is built and published but NO BOX HAS BEEN REINSTALLED FROM IT, and it is deliberately UNVOUCHED. build-golden.sh v3.0.0 baked it with variant V-c and every retargeted assertion passed on the real bake (including mount point mp0 ('/var/lib/felhom'), no mp1 line, both paths are ONE filesystem); it is in the registry (HTTP 200, sha 54e2a4c431daf580…). What has NOT happened is Part 4: reinstall each demo box from it and prove claim → deploy an app → back up → restore	CLOSED — BOTH BOXES REINSTALLED AND PROVEN (2026-08-03)	blocks R-165 reaching PROVEN-LIVE; blocks the capability-map row	The golden is UNVOUCHED ON PURPOSE and that is the safe state, not an oversight: vouching is what makes a fresh install pick it up, so vouching a golden no box has been proven from would put an unproven disk layout in front of the next install anywhere. Prove first, then vouch — the bake script's own output treats the hub record as a separate deliberate step for this reason. Everything else for the merge is shipped and green: controller v0.192.0 (the B2 floor) is live on 9201, agent v0.120.0 is live on BOTH hosts, and felhom-host-install.sh computes the single grow from the thin pool. So a reinstall is now a self-contained piece of work with no code left to write. Order matters: ONE box at a time, demo-hp first, proven end to end, and only then demo-felhom — two in parallel leaves no working reference to compare against. Note demo-felhom carries the PBS-DR/offsite tier, so it is the one whose backup chain a reinstall actually disturbs. ID established free: grep -ro "R-178\b" documentation/ *.md → 0 hits. CLOSED 2026-08-03 — both boxes reinstalled from the merged golden, by two DIFFERENT supply paths, and proven end to end (REPORT.md). demo-hp — the layout proof, installed with --golden local:backup/vzdump-lxc-9100-2026_08_03-07_33_00.tar.zst (installer v1.22.0, sha ed02acb2…, byte-identical to the repo copy): mp0 …mp=/var/lib/felhom,backup=1,size=70G, no mp1; /var/lib/docker → …disk--1[/docker] and /mnt/sys_drive → …disk--1[/sys_drive], both real mounts, both writable, both in /etc/fstab; ONE df figure (69G/65G) and stat -c %d = 64519 on all three paths; reboots 3/3 (08:13:33 / 08:13:59 / 08:14:19, controller healthy in 12s/7s/7s, all three still mountpoints after each). demo-felhom — the pipeline proof, installed with --force-gitea-golden and NO local golden used (preflight logged "golden: none local — will fetch + verify from Gitea in step 7/8", bypassing the 06:58 bake artifact sitting on the same box): verified sha256 54e2a4c431daf580… matches the hub manifest for the golden and verified sha256 a7763d31b55b5ce7… for the agent — the observable this second box exists to produce; mp0 …size=250G, grep -c '^mp1:' → 0, one df figure (246G/233G), reboots 3/3 (09:18:55 / 09:19:12 / 09:19:30). Journey proven on BOTH, endpoint-level (no browser on DooPlex — the exact endpoints the dashboard's own JS calls): claim (POST /claim with the pre-auth HMAC CSRF + felhom_claim_csrf cookie; gate discriminator flipped dashboard not yet claimed → authentication required) → deploy (POST /api/stacks/<n>/deploy) → capture (POST /api/debug/backup/dbdump, which runs the production RunDBDumps) → restore (POST /backup/restore): a planted marker deleted from the live volume came back with an identical sha256 on each box (ac1faae6…ae861 privatebin/demo-hp in 9.2s; bc550798…b59e939 opengist/demo-felhom in 9.4s), recovery units on the single volume in both cases. Ceiling gone, measured: 65 GiB (demo-hp) and 233 GiB (demo-felhom) available to a recovery unit, against the 19 GiB and 45 GiB their pre-wipe mp1 slices offered. Two deviations, both the operator's call and both recorded: the golden was ALREADY vouched when the session opened (hub log 2026/08/03 07:23:26 Artifact manifest set: agent=0.119.0 golden=0.192.0, ~10 min before the first read of this session — so §7's prove-then-vouch order was already spent and the operator elected to accept it); and agent 0.120.0 had never been published, so the vouched agent was 0.119.0 — published + vouched before the reinstalls (→ R-115 third instance). Three new findings: R-179, R-180, R-181	—
R-177	There is no operator-triggerable "run the fill check now" path. fill-watch is reachable only on its daily 03:30 schedule plus the once-at-startup run added in controller v0.191.1 — so the only way to exercise it on demand is to restart the controller	READY (S) — NEW 2026-08-02	—	Noticed while live-validating R-167 on 9201, not by a failure. It cost a controller restart per observation during validation, and it costs the same on a support call: after a customer frees space, nobody can confirm the warning has cleared without restarting their controller or waiting until 03:30. Partially mitigated already — v0.191.2 makes every run log a positive observable (checked N filesystem(s), M unreadable/skipped, K notification(s)), so at least a run that DID happen is visible; the gap is triggering one. The scheduler has GetJobs but no run-now, so this is a general affordance, not a fill-watch one — scope it as "run a named scheduler job now", operator-gated. ID established free: grep -ro "R-177\b" documentation/ *.md → 0 hits	CC
R-173	The hub's SQLite PVC is excluded from every Longhorn backup job. pvc/hub-data carries recurring-job-group.longhorn.io/default: disabled, and backup-daily + backup-weekly (04:00 / Sun 05:00) are the ONLY recurring jobs and both target the default group — so the 128 MB /data/hub.db has no volume-level backup. That database holds host_recovery (every managed box's break-glass root password), host_escrow + host_escrow_superseded (escrow custody), host_pbs_secrets, customer_configs, dr_recipe and the wg endpoints/peers — i.e. the material several documented recovery routes depend on	READY (M) — NEW 2026-08-02	—	Noticed while checking the blast radius of the R-172 WAL change, not by a failure — the WAL work needed to know who copies this file, and the answer turned out to be nobody on a schedule. Establish before designing: (a) whether the exclusion is deliberate (a 1 Gi RWO Longhorn volume snapshotting a 128 MB SQLite file is cheap, so the label looks like a leftover rather than a decision) and by whom; (b) whether anything else backs it up out-of-band that this census missed — the _recovery-inventory-2026-07-28.md records a MANUAL hot copy, which is not a backup. When it is designed, it must be WAL-aware (R-172): a volume snapshot of a live WAL database is crash-consistent and replays on open, which is fine, but any file-level copy must take hub.db-wal too or it silently loses the newest writes. Grep establishing the ID was free: grep -ro "R-173\b" documentation/ *.md → 0 hits	CC
R-158	A local Tier-1 app-data backup failure reaches no hub channel.	CLOSED BY R-167 — SHIPPED + PROVEN-LIVE (controller v0.191.0 + hub v0.89.0, 2026-08-02)	—	Closed by the wire it named; no second row was filed for it (R-167 subsumes and widens it). New unitNotify seam + SetUnitNotify beside the manager's existing three, called from captureAllRecoveryUnits per app with the loop continuing, carrying the target filesystem's used/free bytes at the moment of failure — the cause is usually a full filesystem and those numbers answer why without an operator logging in. ROUTED TO THE OPERATOR, NOT backup_failed, AND THAT OVERRIDES THIS ROW'S OWN PROPOSAL. The proposal above said "emitting the existing backup_failed"; that type carries a customerMessages entry AND sits in settings.DefaultEnabledEvents, so it would email the customer in Hungarian about a failure they cannot act on — precisely the mistake R-97a avoided by minting whole_guest_backup_failed. Decision D-c routes it to the operator and D-c wins. New recovery_unit_capture_failed in allowedEventTypes and notify.operatorOnlyEvents; notify.IsOperatorOnly added so ONE test pins both registers (allowlisted-but-not-operator-only is invisible when they are checked separately — the v0.78.0 defect). Red-proof: removing the register entry shows the customer being emailed. Live on 9201: two events accepted and stored, `operator	sent, and the positive observable customer
R-159	wishlist's data landed in an ANONYMOUS volume — never backed up, orphaned by a redeploy. The image declares VOLUME /usr/src/app/data; the template mounted wishlist_data:/data, a path the app never writes. ResolveDockerVolumeNames returns <project>_<name> only for volumes declared in the compose file, so DumpAppVolumes never sees an anonymous one. Survives a restart, loses on redeploy, never in a backup — harder to notice than papra's.	SHIPPED (templates/wishlist/docker-compose.yml, 2026-08-02) — filed to record the CLASS	—	The class is open even though the app is fixed: any image VOLUME at a path the template does not mount creates unbacked-up storage silently. immich-server has one today at /data — empty when measured, so nothing is at risk now. Proposed REUSE.md rule: a template must mount every path in its image's Config.Volumes, or state why not. Checkable only with the image pulled, hence the runtime gate	CC
R-160	gramps-web persisted three paths and wrote to none of them. /app/data appears nowhere in the image's environment; the accounts DB (GRAMPSWEB_USER_DB_URI) and the family tree (GRAMPS_DATABASE_PATH=/root/.gramps/grampsdb) both landed in the writable layer. Upstream persists eight paths; the template persisted three, one a phantom.	SHIPPED (templates/gramps-web/docker-compose.yml, 2026-08-02)	—	Severity above papra's, and worth keeping visible: papra loses documents the customer may hold elsewhere; gramps-web loses the family tree — the artefact built inside the app, of which no other copy exists by construction. Evidence: app-catalog-felhom.eu/audits/persistence-sweep-2026-08-02/	CC
R-161	The volume-persistence gate is enforced by CONVENTION, not automatically. The catalog repo has no CI of any kind (.gitea/workflows, .github, drone/woodpecker — searched, none exists).	REDUCED SCOPE — open (operator ruling 2026-08-02)	a second person touching templates	RULED. Both obvious enforcement points were rejected for measured reasons. Controller-side at template load: rejected because such a check can only read the file, and a static audit of all 53 templates reports the catalog clean including papra — it would pass on the exact defect it exists to catch; the property is decidable only at runtime. CI: rejected for now — neither repo has any, and there are no users yet. SHIPPED instead (app-catalog-felhom.eu fd7747d): scripts/catalog_gates.py, ONE entry point running all three gates, non-zero exit on any failure, mandated in the catalog's CLAUDE.md the way site_gates.py is. Rationale for the record: of this project's gates, the only ones that ever get run are those with a single entry point named in a CLAUDE.md — site_gates.py is run, R-29's three orphans are named nowhere and have stopped nothing. What remains open is only the automatic half: this is convention, run by a person, and that is sufficient while one person touches templates. Revisit when a second does UPDATE 2026-08-02: catalog_gates.py gained --fast (gate 1 only — the network and runtime gates are deliberately NOT in a hook: a push that pulls images and starts containers gets bypassed within a week and the bypass becomes the habit) and .githooks/pre-push now runs it. The automatic half now has a designated successor row: R-168 (Gitea Actions runner). This row stays open at its reduced scope — the runtime gate remains a deliberate periodic run UPDATE 2026-08-02 (second): the automatic half now EXISTS — R-168's runner executes catalog_gates.py --fast on every push to this repo (measured: run #1, image-pin gate OK — 53 templates, with the two runtime gates announced as skipped and their own output absent from the log). This row's original scope — the RUNTIME volume-persistence gate — is deliberately still NOT automatic and should stay that way: CI that pulls 53 images on every push gets disabled. It remains a periodic run	operator
R-162	docker diff is the gate's only witness, and its failure mode is quiet. The gate's power comes from docker diff excluding mounted paths, which makes "in the writable layer" mechanically decidable — an implementation detail of the overlay driver. On a driver where docker diff is unsupported or lies, the gate degrades to the mount-occupancy and writability legs and would not say so.	WATCHING — a limitation, not a defect	—	It fails closed: the canary self-test would stop reporting BROKEN and the gate would then refuse to report at all. What is wrong is the message — it would blame the prober rather than the driver. Revisit only if a non-overlay storage driver ever ships	CC
R-163	mp1 is RETENTION, not staging — and it is sized as if it were neither.	CLOSED by R-165 — the ceiling it describes no longer exists (golden v3.0.0, 2026-08-03)	— (the sizing question is answered; the work is R-165)	Closed, not merely re-framed. This row was the record of a constraint that was to stay open "until the merge lands". It has landed: the golden ships ONE data volume, so there is no separate 20 G area for a driveless app's recovery unit to outgrow, and the free space an app can use is the box's actual free space. What replaced the constraint is recorded on R-165: the bulkhead the partition also provided is now B2's explicit capture floor (controller v0.192.0), and the measured 2× DB-app unit size this row documented is what justifies the floor's reserve being a reserve rather than a working budget. Caveat carried forward, deliberately: no box has been reinstalled from the merged golden yet (R-178), so every box in the field still has the split layout and this row's consequences remain live ON THOSE BOXES until they are reinstalled. Original finding unchanged below	CC
R-164	C2's chain: the DB volume tar cannot be dropped until a SOUND dump predicate exists. The unit carries both a volume tar and a SQL dump; the restore uses both — the dump is authoritative and replayed after the tar so it WINS (F17), with only the DB service up (R-47) — internal/backup/restore_unit.go:262-266. Dropping the DB container's tar would halve DB-app units and close the R-127(b) initdb-skip password trap (restored PGDATA ⇒ POSTGRES_PASSWORD ignored).	BLOCKED — on the predicate	a dump-validity predicate that is not accounts has rows	The obvious gate is DEAD, measured: ValidateDump warns when the accounts table is empty, and that warning was correct — the live DB genuinely had 0 accounts, and seeding one stopped the warning and put the row in the dump. But a fresh appliance legitimately has zero accounts, so promoting that predicate to a gate would block every new customer's first backup. Order: (1) a sound predicate — dump vs live per-table counts, not an absolute expectation; (2) warn→gate; (3) tar-drop. Until (1), the tar is load-bearing — not because dumps are bad, but because nothing can yet prove one is good. Pairs with R-127	CC
R-165	Merge mp1 into mp0 — the dedicated 20 G backup partition stops existing.	SHIPPED — golden build-golden.sh v3.0.0 + agent v0.120.0 + controller v0.192.0 (B2), 2026-08-03. IMPLEMENTED — the LAYOUT is proven live on both boxes (R-178, 2026-08-03); the BULKHEAD'S REPLACEMENT IS NOT (→ R-181)	—	Variant V-c chosen by the operator on MEASURED evidence, not by reading (audits/SPIKE-r165-phase0-2026-08-03.md): one volume at the NEUTRAL path /var/lib/felhom, with /var/lib/docker and /mnt/sys_drive both binds of subdirectories. Three shapes were built and rebooted; all three boot, reboot 3/3, give ONE df figure and keep a container's statfs("/") on the merged volume — the ordering worry that motivated the probe did not materialise. They differ only in which documented guarantee they break: volume-at-/var/lib/docker puts customer backups INSIDE Docker's data-root (so the ordinary "clear /var/lib/docker" reflex destroys every local unit); volume-at-/mnt/sys_drive puts Docker's ENTIRE data-root under /mnt, which the controller container mounts wholesale — measured: it then sees /mnt/sys_drive/docker, falsifying the bootstrap's own scoping claim. V-c breaks neither. P1 answered R-176(a): a pre-merge archive (mp0+mp1) restore-tests clean with mount_parity: ok in 84 s; mountParity was not weakened. P3: the four golden assertions were RETARGETED, never deleted, and each was RUN against a deliberately wrong shape — 8 checks, 8 passed, including a NEW 2b asserting both paths are ONE filesystem (which catches the S2 shape the spike ranked worse than the split) and a new guard for a leftover mp1 (the old "was mp1 excluded?" pattern could no longer match — a guard that cannot match has silently stopped guarding). B2 shipped first, in controller v0.192.0: a two-term capture floor (97% / 1 GiB) in fillwatch's shape, deliberately beyond its critical band so the customer is always warned before a refusal; it refuses per app and never deletes, because nothing here is generational. Golden 0.192.0 is published (registry HTTP 200, sha 54e2a4c4…) but DELIBERATELY NOT VOUCHED — vouching is what makes fresh installs pick it up, and the right order is prove-then-vouch. Remaining: reinstall both demo boxes from it, prove end to end, then vouch → the work is R-178. STATUS SETTLED 2026-08-03, operator ruling: IMPLEMENTED, not PROVEN-LIVE, and the reason is the substantive part. R-178 proved the layout on both boxes past any doubt — one volume, no mp1, both binds real mounts, one df figure, 3/3 reboots each, claim→deploy→backup→restore, and the ceiling's removal measured at 65 GiB / 233 GiB against the old 19 GiB / 45 GiB slices. But B2, which this row records as the bulkhead's deliberate replacement, does not guard the leg that fills the volume — proven live on demo-hp at 06:40:03 and filed as R-181: the floor is consulted ONLY in captureAllRecoveryUnits (recovery_unit.go:328), while runVolumeDumps (backup.go:535) writes the bulk with no floor check at all, and its refusal message's claim "the previous unit is untouched" was measured FALSE. This row's own framing is what makes that gate the status: it says the partition's bulkhead "is now B2's explicit capture floor". Until R-181 closes, the merge has removed a bulkhead and its stated replacement covers the cheap leg only — and post-merge the unguarded leg can reach Docker's data-root, which pre-merge it could not (it could only fill the dedicated 20 G mp1). PROVEN-LIVE when R-181 closes and a fill is re-run	CC
R-166	App state gets a desired/observed model with its own store. Operator decision D-b, 2026-08-02 (CONTEXT.md S-5)	SHIPPED + PROVEN-LIVE (controller v0.189.0, 2026-08-02)	—	Both blocking facts were established at source before any code was written, and the answers changed the shape. (a) Does a crash-safe journal already exist for the in-flight case? YES, twice — internal/quiesce/quiesce.go (marker + Recover, proven on live hardware by Campaign 8 fault 10) and internal/stacks/migrate.go (migration.json + RecoverMigration) — but neither covers the app-data path: DumpAppVolumesSafe stopped and restarted an app with no marker, no journal and not even a defer. So the pattern existed and the coverage did not; backup.AppStopGuard copies the proven shape into its own file (one file, one writer). (b) Is the SQLite store reachable? Irrelevant, and deliberately unused: metrics.db is optional by design (the controller runs with it absent), and operational state must not live in a store designed to be droppable. Shipped: tri-state desired_state in app.yaml written ONLY by the customer's action (API action switch, DeployStack, UpdateOptionalConfig's redeploy branch, .fab import — a 14-caller census established that StartStack/StopStack must NOT be writers); isBootOrphan reads intent instead of len(Containers) > 0; absent means UNKNOWN, never running, so a legacy app.yaml keeps byte-identical pre-v0.189.0 behaviour; running-only backfill. D-b's every-container requirement was already met by aggregateState and was NOT re-implemented. Live on 9201: all three flows (stop survives a restart; a zero-container running app is recovered by name; a legacy app.yaml is skipped and never inferred as stopped). Also fixed en route: SaveAppConfig rebuilt AppConfig field-by-field (the R-100 shape) and would have dropped the new field on every save across nine call sites	—
R-167	Storage monitoring and backup alerts.	CLOSED — SHIPPED + PROVEN-LIVE (controller v0.191.0/.1/.2 + hub v0.89.0, 2026-08-02)	—	Operator decision D-c. It shipped BEFORE the R-165 merge, not with it — D-a's condition (2) says the monitoring lands in the same step and never after, and landing it first is strictly better and costs nothing. Customer half: new internal/fillwatch, per FILESYSTEM (never per app — one full disk holding ten apps would fire ten times). It emits the PRE-EXISTING disk_warning/disk_critical pair, which was allowlisted, copy'd, in DefaultEnabledEvents and checkbox'd with NO PRODUCER IN ANY REPO — a complete customer pipeline with no producer, the sixth built-but-never-wired instance here; minting a new near-duplicate type would have left it inert forever. Two threshold terms, whichever trips first (85% / 5 GiB; critical 95% / 2 GiB) because a percentage alone lies at both ends of this fleet's size range — proven live: the critical crossing fired on the FREE-BYTE term (1.7 GB) at only 91% used. Edge-triggered on escalation, state persisted, hysteresis dead zone at 75% / 7 GiB pinned by a test; a nil usage read never warns and never clears one (§8.4). The hub's two generic customerMessages entries were removed — FormatCustomerEmail prefers the entry over the message, so keeping them would discard the drive label and the byte figures. Operator half: see R-158. Live on 9201, all three flows: disk_warning then disk_critical both `customer	sent` with the Hungarian rendered, exactly two events across three boots (the edge trigger held on the one between), then a silent clear that re-armed. v0.191.1 added the once-at-startup run (Daily/Every both wait for their first tick, so a box BOOTING over the line would have stayed silent up to 24 h — the R-100 shape); v0.191.2 added a per-run positive observable, earned when a quiet run during this session's own validation proved unreadable as evidence. Follow-ups: R-177 (no run-now path)
R-168	CI: no runner exists, and with trunk-based pushes CI can DETECT but not BLOCK	SHIPPED — and the alarm is DEMONSTRATED (2026-08-02)	—	Runner live: homelab-manifests/gitea-system/act-runner.yaml, an unprivileged host-mode act_runner in gitea-system, one owner-scoped registration serving all four repos (measured: tasks 7-10 all claimed by felhom-gates-runner). .gitea/workflows/gates.yml in each repo runs that repo's entry point with --fast and nothing else; no uses: step anywhere. Six probes, all answered, none STOPped — audits/SPIKE-ci-runner-2026-08-02.md. The two that changed the design: P2 (stock image has git but NO python3 → custom image felhom-act-runner:0.1.0, base pinned, python3 and nothing else) and P6 (a runner that loses /data/.runner re-registers and leaves a dead record behind → the PVC is load-bearing, measured both ways). P5 is the one that mattered: a failed run produced NO mail, NO notification row and NO log line from Gitea, so the run now sends its own alarm via Resend and prints the provider's accepted id. Proven end to end, not asserted: a deliberately broken commit pushed with --no-verify → run #6 failure → RESEND-ACCEPTED id=5ff34766-c5f8-4588-8104-08296aeb45ab. Posture shown from the live pod spec: privileged: false, all caps dropped, no docker socket, no hostPath, automountServiceAccountToken: false, sized at half Gitea's limits so it cannot crowd out the service holding every repository on the same node. The standing limit stays true and is written into the manifest and every workflow: it DETECTS, it does not BLOCK — making it block is → R-169	—
R-169	CI can only report, because there is no gate in the road. Every felhom repo pushes straight to main with no pull request, so there is no merge for a status check to stand at. R-168's runner therefore notices a broken push after it has landed	WAITING-ON-OPERATOR (a working-style decision, not a defect)	an operator ruling	Making CI blocking requires two things this task deliberately did NOT do, because both change how the operator works and that is not a task's call: (a) branch protection on main, and (b) a pull-request workflow instead of direct-to-main pushes. The cost is real — every change would need a PR, which for a single-operator project may be worse than the disease. The current arrangement is two nets, and it is not nothing: .githooks/pre-push REFUSES locally, and R-168's runner NOTICES when that hook was skipped or was never armed in a clone, and emails. The honest gap is the window between a --no-verify push landing and the operator reading the alarm. Decide only if that window ever actually costs something	operator

Why the TOP READY rows rank this way

This covers the next few only — it is deliberately not a full ordering of the table above, so that there is one ranking to maintain rather than two.

1. R-95 — the largest data exposure: the tier holding the customer's documents and photos is the one whose credential can delete. The snapshot mitigation is now armed (daily 00:00, keep 7), but it has taken zero snapshots so far and it does not touch the root cause — the box can still forget --prune its own repo.
2. R-94 — de-ranked 2026-07-29. The prior rationale ("until it moves every hub-driven install gets the pre-R-82 default") was false: the constant selects no script and every install already fetches 1.22.0. What remains is a wrong label plus two pieces of dead safety equipment — a drift gate nobody runs and a test that compares a constant to itself. Cheap and worth doing; not high-consequence, and it blocks nothing.
3. R-86 — CLOSED 2026-08-03, agent v0.121.0 + hub v0.91.0, proven live on demo-felhom.
4. R-87 — re-ranked UP: R-86 built most of what it was waiting for (per-archive due-ness, a proof that names its archive, and a staleness window that learns a tier's rhythm). What is left is restic-specific — there is no scratch-guest analogue — so it still needs its own design, but it is no longer waiting on a scheduling model that did not exist.
5. R-185 — CLOSED 2026-08-03, agent v0.123.0 + installer 1.24.0, proven live on both demo boxes. The silence was fixed as well as the grant: the box now asks whether it may READ each tier it depends on, because an empty listing cannot distinguish forbidden from newborn.
6. R-189 — CLOSED 2026-08-03 with R-188 and R-186, agent v0.122.0. The three reporting/release signals that misreported their own work are fixed; R-185 is the one that remains open from that group and is untouched by this — it is a missing storage ACL on demo-felhom, not a reporting defect.
7. R-110 — last because it is not a READY row: the ruling is the operator's, not CC's, and there is nothing for CC to build until it lands. Ranked here rather than omitted because it is the only item on this page about the publish channel of the most privileged artifact Felhom ships, and today's exposure is zero — which makes now the cheapest moment it will ever be to decide.

The 2026-08-02 intake (R-156 … R-164), ranked

Filed in one pass from Campaign 10, its two spikes, and the 53-template catalog persistence sweep. R-156 and R-157 had lived only in audit documents — the identical "minted in a spike doc and never carried across" failure the register already records for R-153/R-154/R-155, caught by the sweep's own §8.0 while it was happening. R-158 was minted by a second session on the same day for an unrelated finding, which is why the sweep's proposals were renumbered to R-159…R-162 at filing time.

1. R-157 — highest: a deployed: true app can stay down indefinitely after a power cut or hard reset, and in mechanism B nothing reports it on any channel (0 currently down). It is the only row here where the customer loses service and has no signal at all.
2. R-156 — the class is now detectable and two of three apps are fixed; what remains is papra's referral, one app, well understood. (Promoted 2026-08-02: R-161 was ranked here because nothing ran the gate; it now has a mandated entry point, so R-156's residue is the larger remaining item.)
3. R-163 — a real ceiling that silently caps local backup once an app outgrows mp1, and it gates Tier-2 and Tier-3 as well. Ranked below the above only because overflow itself is safe today — it refuses per app and preserves the last good unit byte-identical. RE-FRAMED 2026-08-02: no longer waiting on a ratio — decision D-a merges mp1 away, so the row is now the record of the constraint and the work moves to R-165 (with R-167 shipping in the same step). R-165 inherits this rank; it is the highest-ranked item that must land before any external install.
4. R-158 — the gap that makes R-163 dangerous: cross the size line and one page tells you. On its own it is a notification gap, not a silent failure, which is why it sits here and not higher.
5. R-164 — blocked on a predicate, no customer impact today; it only becomes urgent if the unit size in R-163 is judged unacceptable, since the tar-drop is the cheapest way to halve it.
6. R-161 — de-ranked 2026-08-02, ruled and shipped at reduced scope. The gate now has one mandated entry point (catalog_gates.py), which is the shape that actually gets run here. What is left is the automatic half, and that is sufficient while one person touches templates — so it ranks low by design, not by neglect. Revisit when a second does.
7. R-162 — WATCHING only. A limitation that fails closed; revisit if a non-overlay driver ships.

R-159 and R-160 are SHIPPED and are not ranked; they are filed to record the class, and R-159's class (an image VOLUME at an unmounted path) is still live — immich-server has one today.