Two boxes, two DIFFERENT supply paths, so the session proved the disk shape and
the delivery route rather than one of them twice.
demo-hp (layout proof, --golden <local volid>): mp0 at /var/lib/felhom,
backup=1, 70G, no mp1; /var/lib/docker and /mnt/sys_drive both real mounts of
its subdirectories via fstab; one df figure and one device id (64519) on all
three paths; reboots 3/3 with the binds surviving each.
demo-felhom (pipeline proof, --force-gitea-golden): fetch_verify succeeding
against the vouched manifest for BOTH artifacts -- 'verified sha256
54e2a4c431daf580... matches the hub manifest' for the golden, a7763d31... for
the agent. 250G single volume, grep -c '^mp1:' = 0, reboots 3/3.
Journey proven on both, endpoint-level: claim -> deploy -> back up -> restore,
with a planted marker returning byte-identical on each box. Ceiling measured
gone: 65 GiB and 233 GiB available to a recovery unit, against 19 and 45.
R-165 -> IMPLEMENTED, not PROVEN-LIVE, on the operator's ruling. B2, which that
row records as the bulkhead's replacement, fired live for the first time and
does refuse per app, delete nothing and alert -- but it is checked only in
captureAllRecoveryUnits while runVolumeDumps writes the bulk unguarded, and its
'the previous unit is untouched' claim was measured false (182,272 B dump
replaced by 2,147,666,432 B under a manifest still dated 06:34:26). -> R-181.
New: R-179 (uninstall leaves NAS network-storage units), R-180 (--archive-storage
not cross-checked against the ACL grant; 403 at step 8/8 after root@pam is
rotated), R-181. Third instance of R-115 recorded (agent 0.120.0 unpublished).
No code written, no version bumps -- this was a runbook.
07-backup-architecture.md gains §7.5.1 (S-1: the contract changed in the same
session): the ceiling §7.5 describes no longer exists for a box built from
golden >= 0.192.0, the bulkhead's replacement is recorded, and R-175 is FIXED
here rather than left standing — the bound is restated as a function of mp1
and scoped to split-layout boxes, naming all three real shapes.
Capability map: new row as IMPLEMENTED, deliberately NOT proven-live, with
the missing leg named — no box has been reinstalled from the golden, and
"the golden baked" is not "a box built from it works".
R-163 CLOSED: the ceiling it recorded stops existing. R-176(a) answered by
P1; (b) WITHDRAWN, since every node is reinstalled rather than migrated.
R-178 filed for the reinstalls, which were not done this session.
CONTEXT S-13 (the variant chosen on measurement; pruning rejected with its
reason) and S-14 (prove first, then vouch — the golden is published but
deliberately unvouched, because vouching is what makes a fresh install pick
up a layout no box has been proven from).
STATUS: plain-language section; both operator questions now answered, so the
waiting-on-you item is cleared. Two older entries trimmed so the page did
not grow.
R-167/R-158 shipped and proven live (controller v0.191.x, hub v0.89.0):
two new capability-map rows PROVEN-LIVE with live citations, and
07-backup-architecture.md §7.5's closing claim "nothing warns when an app
crosses the line" is now false and rewritten (S-1: an architectural
contract changed in the same session). §7.5 also gains the caveat that its
size bound is ONE BOX'S, not the fleet's.
Part 3 SPIKE (audits/SPIKE-r165-mp1-merge-2026-08-02.md): M1-M5 measured,
NO layout touched. Three findings the merge session must not re-derive:
"the layout" is not one thing (200G/50G vs 50G/20G vs 16G/8G); mp1 is a
BULKHEAD and not only a ceiling, so after the merge an overflow reaches
/var/lib/docker; the golden fails closed on the split in four places.
D-a's condition (1) is currently SATISFIED — no external box is in the
hub's register, and both demo boxes are Tier 0 and reinstallable.
Recommendation given, choice NOT made — it ends at the operator's ruling.
CONTEXT.md S-11 (D-c's routing, and why R-158's own backup_failed proposal
was overruled) and S-12 (the monitoring landed BEFORE the merge).
STATUS.md gains the plain-language section and the merge decision, with two
older entries trimmed so the page did not grow.
New rows R-174 (closed same session), R-175, R-176, R-177; each ID grepped
free before minting.
R-172's root cause was not tuning — the WAL/busy_timeout pragmas had never been
applied, because the DSN used mattn/go-sqlite3 syntax against modernc.org/sqlite,
which ignores unknown parameters without an error. Recorded that way so nobody
re-reads it as "SQLite was slow".
R-173 NEW: while establishing who copies hub.db for the WAL change, found
pvc/hub-data labelled recurring-job-group.longhorn.io/default: disabled, with
backup-daily and backup-weekly the only recurring jobs and both on the default
group — so the hub database has no volume-level backup, and it holds every box's
break-glass root password plus the escrow custody records. Filed, not fixed:
whether the exclusion is deliberate is an operator question.
The session report is REPORT-r172-hub-wal.md, not REPORT.md, per the
parallel-session rule — REPORT.md belongs to the controller session that ran
immediately before this one.
It also records, plainly, that a 60-concurrent load test I ran OOM-killed the hub
pod three times against a 256Mi limit. Not the WAL change, and not a test I
should have run against a Tier-2 box; the unit tests already proved the property.
Controller v0.190.0. Docs only here; no hub change, no hub version bump.
- audits/DIAG-bootrecon-drive-absent-2026-08-02.md — NEW. The Part 0 diagnosis,
including the run that produced a FALSE NEGATIVE and the mechanism behind it
(the agent re-binds an unmounted drive within ~60s, so the drive gate's startup
reconcile restarted the apps one second before the sweep looked). Records that
the write hazard was blocked only by an ACCIDENTAL filesystem permission that no
code owns and no test pins.
- architecture/02 §0a — the boot-recovery contract (S-1): both gates read desired
state; the sweep observes a SETTLED fleet and each sample must refresh first;
nothing is started without asking, fail-safe. Plus the durable warning:
Manager.StartStack has no gate of its own.
- 00-capability-map — the boot-recovery row, with the repeat count cited per N.5
(6 of 6 hard resets) rather than a bare PROVEN-LIVE.
- OPEN-ITEMS / ROADMAP — R-157 CLOSED (both mechanisms), R-170 CLOSED, R-171 NEW
and closed the same session, marked a regression from v0.189.0.
- STATUS.md — the power-cut line moved from "What's broken" to "What works right
now" with its repeat count; one dated bullet in the change log.
- CONTEXT.md S-13 — the lessons worth carrying: "it didn't happen this time" is
not a disproof; widening a window makes previously-unreachable overlaps
reachable; and a settle detector is only as good as the freshness of what it
samples — the fix's own defect, found live rather than by review.
Controller v0.189.0 implements operator decision D-b. Docs only here; no hub
change and no hub version bump.
- architecture/02-controller-module-map.md §0a — NEW, and it is the S-1 contract:
desired (app.yaml) / in-flight (own marker file) / observed (not persisted),
with the rule that ties them — never derive one from another. Absent desired
state means UNKNOWN, never "running". One file, one writer. D-b's binding
safety rule quoted verbatim.
- 00-capability-map.md — the boot-recovery row now rests on a recorded signal,
with the three live flows from 9201. The interrupted-operation half is marked
IMPLEMENTED, not PROVEN-LIVE: nobody killed the controller mid-backup on metal.
- OPEN-ITEMS/ROADMAP — R-166 SHIPPED with both blocking facts and their answers;
R-157 mechanism B CLOSED and A restated as the whole item; R-170 NEW (the
drive-backed boot gate still infers a Stop from a container count).
- STATUS.md — the "an app can stay switched off and nothing says so" line
rewritten to what is actually left: timing.
- CLAUDE.md — end-of-session checklist gains: confirm your own last push's CI run
went green, BY RUN ID. The failure email is a push signal; this is the pull check.
- CONTEXT.md S-12 — the rulings, and the two lessons worth carrying: a test that
constructs the thing it should prove the caller constructs is hollow (its
red-proof will say so), and a field-by-field struct rebuild in a save path is a
defect on sight.
SPIKE-ci-runner-2026-08-02.md: all six probes with method, measurement and ruling; none
STOPped. P2 (stock image has git but no python3) and P6 (a runner that loses its state
re-registers and orphans the old record) changed the design; P5 (a failed run signals
NOTHING) is why the alarm exists at all.
R-168 SHIPPED with its evidence. R-29 CLOSED — on the demonstrated alarm and not on a green
run, as required: the class it opened is answered at both ends, the hook refusing locally and
CI catching a --no-verify bypass and emailing. R-161 noted: its automatic half now exists for
the STATIC gate, while its original scope, the runtime gate, is deliberately still not
automatic and should stay that way.
NEW R-169 (grep established R-168 was the highest in use): CI can only report, because there
is no gate in the road. Making it blocking needs branch protection plus a PR workflow, both
of which change how the operator works — so it is theirs to decide, and the row states the
cost honestly rather than recommending it.
CONTEXT gains S-8 (CI detects, does not block, and why that is structural), S-9 (a detector
that tells no one is not finished, plus the curl and Cloudflare-1010 traps), S-10 (the runner
is unprivileged because DooPlex is Tier 2), S-11 (CI reproduces the sibling layout).
CLAUDE.md gains the rule earned by red-proofing: a go test -run pattern that matches no test
prints ok and exits 0, and an instrument that can silently drop results is not a measurement.
REPORT overwritten per the standing rule; every red-proof, the core.hooksPath probe's four
measured outcomes, Scenario C's refusal-and-bypass, the hub deployment and the live Setup-tab
read are recorded there, plus three observations and two deliberate deviations from the spec
(a comment-only edit to felhom-host-install.sh, and __pycache__ in .gitignore).
STATUS: the 'check that needs a person to remember it' line is rewritten — the checks now run
themselves before every push, with both honest limits stated in plain words; and one entry
records the thirteen-check cleanup and the deleted installer version number.
Documentation only — no code, no box, no build.
STATUS.md (repo root, 652 words / 67 lines): what works · what's broken ·
what we're working on · waiting on you · changed since. A VIEW of
OPEN-ITEMS.md, holding nothing of its own; not CONTEXT.md, and both files
now say why they stay separate. No R-n is the subject of a sentence —
identifiers are bracketed pointers only.
CONTEXT.md S-5 records the four operator decisions taken 2026-08-02
(D-a … D-d), none of them implemented:
D-a merge mp1 into mp0 rather than resize it — before any external
install, and D-c ships in the same step → R-165
D-b desired/observed app state in its own store, with the state-store
safety rule verbatim → R-166 (BLOCKED)
D-c customer fill warning + operator backup-failure alert → R-167
D-d only DooPlex and Peti's box are protected → target-selection.md
R-163 RE-FRAMED, not closed: the sizing question is withdrawn rather than
answered; the row survives as the record of the constraint until R-165
lands. R-156's papra referral RESOLVED — deployed nowhere, so the template
fix strands nothing; the docker ps evidence is recorded with its
provenance and its scope limit.
target-selection.md: two protected machines, everything else disposable.
ep0 is no longer Tier 2 but is not scratch (it holds the only off-premises
copy of real customer data) — flagged for explicit operator confirmation.
The demo-box backup-target fence drops from prohibition to stated cost,
because D-d spends that reference anyway.
CLAUDE.md gains an End-of-session checklist carrying the STATUS.md
maintenance rule and "a finding goes in OPEN-ITEMS.md first".
admin