Commit Graph

23 Commits

Author SHA1 Message Date
admin 3f2b7bc023 RECON: trace the offsite DR chain link by link — it does not join up (R-198..R-201)
gates / gates (push) Successful in 7s
Read-only recon of the escrow -> recovery chain, from a dead node to an open
repository. No production code, no build, no version bump.

Headline: the hub's superseded-escrow retention does NOT retain the offsite
repository password. host_escrow_superseded has no identity_blob column and
demoteCurrentEscrowTx copies only the K-escrow blob, so what survives a
supersession is the PBS datastore key, not the restic repo password. The next
escrow ceremony -- which the system tells a rebuilt box's customer to run --
destroys the last copy. Both demo boxes crossed that line on 2026-08-04.

Also established:
- the hub's blob-serving endpoints (re-enroll / restore-directive) have zero
  callers anywhere: agent, hub UI, scripts, runbooks (R-199)
- POST /backup/offbox/inject-password is routed and handled but no template
  contains the form (R-200)
- nothing in the recovery path has ever been exercised; the one live
  round-trip proof (2026-06-10) predates the ResticRepoPassword field (R-201)
- a fail-closed mint refusal IS implementable: the report ACK already carries
  escrow{identity_blob_present, restic_pw_sha256} and the controller discards
  it whenever no offbox target exists

Corrections: yesterday's spike annotated (candidate (b) overturned in part --
unattended recovery is impossible, customer-present is not); capability-map
retention claim struck through and replaced with what the code does.

Deliverable: documentation/audits/RECON-offsite-dr-chain-2026-08-04.md
Register: new R-198..R-201; R-193 and R-192 updated; STATUS.md refreshed.
2026-08-04 12:16:04 +02:00
admin b7c5cfc061 docs: R-193 spike report, operator STATUS, and the ROADMAP R-1 scope correction (F3 covers HOST re-enroll, not a guest rebuild)
gates / gates (push) Successful in 8s
2026-08-04 11:12:08 +02:00
admin a684a81796 docs: R-191 closed, R-190 mitigation shipped (mechanism open), R-194 filed
gates / gates (push) Successful in 7s
R-191: the gate was run first and it mattered — ep0's prune jobs have run daily
since 2026-07-27 (18 tasks, all OK). THREE of my own queries said the opposite and
all three were broken instruments (worker-type vs worker_type, prune vs prunejob,
proxmox-backup vs proxmox-backup-proxy); acting on any would have disabled the only
pruning attempt while reporting that nothing prunes.

R-190: the row now reflects the MITIGATION, not the cause, and says so. Includes
that v0.124.0's transition record failed in production while every unit test
passed, and Part 3's single pass — the token-rotation class is real and documented
in our own installer but does not fit this box (same token still authenticates, the
other three grants survived, no installer run evidenced); ruled out on this box, not
fleet-wide.

R-194: PVE caches permissions (~40s and ~16min measured), so a single permission
read is a lagging indicator and may explain when R-190's refusal surfaced.

07-backup-architecture: the offsite retention row was RIGHT and the configuration
disagreed with it — noted, since a doc that states a contract does not enforce it.
2026-08-04 10:04:25 +02:00
admin 33f0ab2f33 R-193: demo-hp offsite restored on the box; escrow correction recorded
gates / gates (push) Successful in 7s
Re-issue through the designed endpoint: fresh password staged 07:11:51, settle-gate
GO, consumed 15s later at 07:12:06, offsite configured at 07:12:09 against the SAME
sub-account (275124) and repo path. Both boxes now escrowed.

CORRECTION to this session's own reading: the escrow did NOT recover by itself.
escrow_state went pending -> escrowed 15s after the apply and I inferred automation;
the operator had run the ceremony. It needed a human on both boxes — demo-hp at
07:16:02, demo-felhom at 07:20:28 (its escrow had been pending since its own rebuild
the day before). A 15-second state change is not evidence of automation, which is the
same class as reading an absent log line as success.

Snapshot continuity is NOT established: both boxes report snapshot_count 0, but the
run-history keys are ABSENT rather than zeroed — a controller that has never run an
offbox backup, not an empty repo. The 04:15 run decides it.
2026-08-04 09:23:08 +02:00
admin 2584dfb938 R-193: a guest rebuild silently drops the offsite tier; R-192 cause established
gates / gates (push) Successful in 8s
Operator confirms no hub-side offsite config change, so the regression was not an
action. Evidence: demo-hp's controller went 0.187.0 -> 0.192.0 at 06:12:18 with a
new config hash and the agent re-keyed its leaf three minutes earlier — a guest
rebuild. The last pre-rebuild report shows the tier fully healthy: escrowed, last
success 02:16:39Z, 15 snapshots, 40.9 MB. No offsite object in the 108 reports
since.

Mechanism: the restic credential is delivered once. demo-hp consumed its secret on
2026-07-23; the rebuilt controller has no copy and no way to request another.
demo-felhom survived the SAME rebuild only because its secret was still unconsumed
— it consumed it four seconds after its config hash changed and was reporting
offsite again 76 seconds later. That difference was luck, not design.

Also sharpens R-192: the self-heal's guard refuses when any report since the
consume carried an offbox target, but that query reads the OLDEST 500 reports —
all of which predate the rebuild. Healthy history before a rebuild is not evidence
the credential still works, which is why the automation that exists for this case
declined to act.
2026-08-04 09:05:53 +02:00
admin 185661062a STATUS: back to one screen
gates / gates (push) Successful in 7s
2026-08-04 08:58:04 +02:00
admin 967ddf2f44 R-192: offsite_delivery_stuck states the opposite of what it measured
gates / gates (push) Successful in 7s
demo-hp has been emailing daily since 2026-08-03 06:12 UTC that '500 reports
since carry no offbox target — the credential is likely burned'. Measured against
the hub's own data: all 500 of those reports DO carry one. The credential was
consumed 2026-07-23 and applied successfully; the controller reported an offsite
object continuously until 2026-08-03 05:59:21, then it vanished at 06:12:19 and
has been absent for 108 reports. A regressed apply, not a burn.

Two defects. maybeEmitStuck builds its message from ReportsSinceConsume while
hardcoding 'carry no offbox target' and never consults OffsiteReportsSinceConsume,
so the recommended action targets a failure that did not occur. And maybeHeal
refuses SILENTLY on exactly that field ('regressed-apply shape -> operator's
call', a bare return), so there is a daily email with the wrong story, no heal,
and nothing saying why the heal declined.

The underlying condition is real: that box currently has no off-site copy of its
customer app data, since 08:12 CEST on 2026-08-03. What removed it is not
established.
2026-08-04 08:57:53 +02:00
admin d692c46db0 RUNBOOK: the host tier is restore-PROVEN unattended on both demo boxes
gates / gates (push) Successful in 7s
Four SCHEDULED runs, none triggered by hand: demo-felhom host 83.8s / offsite
540.4s; demo-hp host 109.3s / offsite 300.1s. Each restored into a scratch guest,
booted, verified and destroyed itself; zero 990000 guests or volumes afterwards
and both local-lvm figures returned to their pre-run values.

Both boxes had BOTH tiers due at once, so R-86's ordering was observed live for
the first time: never-proven sorted first, each box took its HOST tier, deferred
the offsite one, and picked it up on the next evaluation six hours later. The
host-tier proofs reached the hub through R-189's merge — demo-felhom's report
carries two tiers, and the local one can only have come from disk.

The capability map's optimistic half is cashed, with its scope stated: these two
boxes, not the fleet.

Surfaced and filed rather than fixed:
- R-190: a storage ACL that demonstrably worked at 04:44 was gone by 09:24, with
  a reinstall, any logged pveum activity and any cluster-log entry ruled out.
- R-191: every weekly offsite backup uploads successfully and then fails the job
  on a prune the box is deliberately not allowed to do (R-89 moved it
  server-side; both boxes still arm keep_last=2).

Two corrections to yesterday's record: the R-185 drift DID surface as 403s on the
write path (six, with the hub raising whole_guest_backup_failed at the first), and
my earlier "no restore_test_* events" was produced by grepping a 404 page.
2026-08-04 08:13:43 +02:00
admin e3187c86d5 docs: R-185 closed — the silence as well as the grant
gates / gates (push) Successful in 8s
- OPEN-ITEMS: R-185 closed with the measurement, the corrected root cause (the
  installer's Scenario-F reuse arm, not PVE_STORAGES), and the live sequence.
  Records that demo-hp carried the same drift and was fixed too.
- capability map: the whole-guest row's HOST-tier half was OPTIMISTIC and now
  says so — that tier was not merely unproven, it was unprovable on both demo
  boxes, and every live proof cited was on the offsite tier.
- vzdump-target-move runbook: its item 5 predicted this; annotated (not
  rewritten) with what actually happened — the create arm did grant, the reuse
  arm did not, and it surfaced as a silent unreadable tier rather than the 403
  the item expected, because vzdump writes through a root path.
- CONTEXT: S-21 (an empty listing cannot distinguish forbidden from newborn; the
  measured trap that an ungranted path answers with INHERITED privileges) and
  S-22 (the Scenario-F arm must finish the job).
- STATUS: rewritten for the operator, back to one screen.
2026-08-03 19:02:25 +02:00
admin 81d4438630 docs: R-189, R-188 and R-186 closed (agent v0.122.0); R-185 stays open
gates / gates (push) Successful in 8s
- OPEN-ITEMS: all three rows closed with their measurements. R-189 records the
  migration consequence seen live (a pre-R-189 record has no tier and is not
  reportable, so upgrading does not retroactively make an old proof visible).
  R-188 records both directions demonstrated and the fixture cleaned up. R-186
  carries the before/after sha values including the CGO_ENABLED discrepancy.
  The priority list states explicitly that R-185 is the one left open from that
  group and that this session did not touch it.
- capability map: the restore-proof row now says the evidence path itself had a
  gap — the proof reached the hub only because no restart intervened — and what
  closed it.
- CONTEXT: S-19 (a proof is durable and reportable, a failure is neither, and
  why) and S-20 (the release ordering, what each step protects, and how a binary
  is verified).
- STATUS: rewritten for the operator, back to one screen.
2026-08-03 16:46:49 +02:00
admin c102832892 docs: R-86 closed and proven live; ep0 recorded as protected; R-185/186/187 filed
gates / gates (push) Successful in 8s
- OPEN-ITEMS: R-86 CLOSED with the trap in its own wording recorded (the literal
  reading is never true on a daily tier); R-87 re-ranked UP because R-86 built
  most of what it waited for; R-185 (the agent cannot list demo-felhom's host
  backup tier — a missing storage ACL, pre-existing), R-186 (a released binary's
  sha is not reproducible from its tag), R-187 (R-115's publish leg had never
  actually run) filed. R-184 was the highest ID in use.
- ROADMAP: R-86 collapsed, keeping the reasoning and correcting the shape the row
  itself proposed — which would have been the never-fires version.
- 07-backup-architecture: new contract section — restore-testing is per ARCHIVE
  GENERATION, with the trap and what did not change (S-1).
- 00-capability-map: the unattended restore-proof row upgraded to PROVEN-LIVE on
  the 635 s due-triggered offsite run, with the restart and teardown evidence.
- CONTEXT: S-17 (the rule, the trap, the config key, the hub's derivation) and
  S-18 (ep0 is Tier 2 — extends D-d's protected list to three machines).
  Numbered 17/18 because S-14 and S-15 were already duplicated in the file.
- STATUS: rewritten for the operator, trimmed back to one screen.
2026-08-03 15:28:32 +02:00
admin e34b614e5b docs: R-182 closed, R-90 closed on measurement, R-86 unblocked, ep0 record corrected
gates / gates (push) Successful in 7s
R-182 CLOSED (controller v0.194.0 + hub v0.90.0/.1), proven live on demo-hp. The
hub's notification_log for the run reads: two per-app failures RECORDED, one
digest SENT naming both, and the customer channel SKIPPED with operator_only.
Against the measured previous behaviour — two failures, one email naming one
app, one leaving no trace anywhere.

Scenario D proved itself on an event I had not planned: disk_critical alarmed on
two filesystems, the second was collapsed by the cooldown, and that collapse is
now visible WITH ITS KEY. Yesterday it would have left nothing at all.

A gap the spec did not anticipate is recorded with its fix: the per-app event
also fires from the periodic sweep, outside any run, so making it record-only
would have created a NEW silence. The sweep emits a digest too, with no run_id,
so it stays under the ordinary hourly cooldown.

ep0: MEASURED on the box — 7757 MB (8 GB), 4 vCPU, and the 4 GiB swapfile
SURVIVED the resize and is active (checked, because a resize is a stop/start).
The 40 GB local disk is UNCHANGED, so no disk figure was touched anywhere.

Five documents corrected — three of which the task's list did not name, found by
searching. Two audit/evidence documents ANNOTATED, body untouched: they record
what was true when written and that is their value.

R-90 CLOSED. R-86 unblocked and re-ranked, stated honestly: 8 GB is comfortable,
not unbounded — the original OOM was a 14.46 GB restore — so the restore-test
cadence should still be paced, just not by fear of the endpoint.

target-selection.md's "D-d did not name ep0 either way" is deliberately left
standing. It is the operator's question, not CC's.

STATUS.md 127 -> 83 lines, items rather than sentences.
2026-08-03 14:00:54 +02:00
admin b0b269b28d docs: R-110 + R-115 closed, R-182 re-scoped by measurement, R-183/R-184 filed
gates / gates (push) Successful in 7s
R-182's direction REVERSED by Part 0's measurement. Filed yesterday as "the
reserve re-alerts on every status refresh" — too many alerts, seen at the
sending end. Measured at the receiving end: 9 events received today, 2 operator
emails sent. When two apps are refused in the same second the operator is told
about ONE; the other is dropped before LogNotification, so it leaves no row on
any channel and cannot be audited. The operator cooldown key is
customerID:eventType(+tier) and the capture-failed event carries `app` but no
`tier`, so the key has no app identifier. Same failure mode as R-97a, in a
second event type that never opted into the narrow fix. Nothing changed —
Part 0 was investigation only.

Correction owed: yesterday's report said "one recovery_unit_capture_failed per
app, HTTP 200". True of what the CONTROLLER pushed; a reader would take it as
"the operator was told about each app", which is false.

R-110 CLOSED (installer v1.23.0). Both channels moved. The spec's mechanism for
channel 2 rested on a factual error — the run-time fetches are sixteen, not
nine, and come from felhom-agent, not this repo — so no tag here could cover
them; pinned to the agent version being installed instead, on the operator's
ruling. Channel 3 needed no change: the URL never carried a ref, so no hub
change and no hub bump.

R-115 CLOSED. release-agent.sh builds, tags, publishes and verifies by an
independent download; check-published-versions.py refuses a tag with no package;
CI now runs the full gate set so it actually runs.

R-183 NEW+CLOSED: a fresh install fetched the vouched agent binary and its
sixteen config files from two different refs, and nothing compared them.

R-184 NEW: nothing stops the hub vouching a version that was never released.
The R-115 gate cannot see it — measured, the hub manifest and Gitea's package
listing are both 401 anonymously.

capability map: new PROVEN-LIVE row for the published installer channel.
STATUS.md 138 -> 127 lines.
2026-08-03 12:44:08 +02:00
admin fb652024ea docs: R-181 closed, R-156 closed, R-110 + R-115 rulings recorded, R-182 filed
gates / gates (push) Successful in 7s
R-181 CLOSED (controller v0.193.0 + v0.193.1) and proven live on demo-hp for
BOTH reserve terms. The reserve is now a per-app, per-run ADMISSION decision
taken before the app's first write and covering all three write legs, and it
gained a size term. The refusal's wording was not weakened; the behaviour moved
so it became true, verified by sha256 tree fingerprint.

R-156 CLOSED — papra's template mounts the app's own data root. Precondition
re-measured rather than inherited (both boxes were wiped today).

Part 4, documentation only, nothing built:
- R-110 WAITING-ON-OPERATOR -> READY. Ruling: option (b), the installer's
  publish channel moves to a TAG. Recorded with the condition that decides
  whether it works at all — it must cover BOTH the /scripts/ git-sync and the
  nine files the installer fetches from raw/branch/main.
- R-115 WAITING-ON-OPERATOR -> READY. Ruling: mechanism (b), a build-side gate
  refusing to deploy or vouch an unpublished version. The third instance (agent
  v0.120.0) would have silently downgraded both demo boxes while succeeding.

R-182 NEW: the periodic status refresh has no admission scope, so a refused app
re-alerts on every poll (measured: a second alert pair 13s after the run's).
Pre-existing in v0.192.0; deliberately not fixed in the R-181 task.

capability map: the local-backup row moves to PROVEN-LIVE in BOTH halves.
ROADMAP: R-165 collapses to CLOSED; R-181 collapsed into it.
07-backup-architecture.md: the reserve's contract stated as what the code
provides (S-1 — an architectural contract changed in the same session).
STATUS.md trimmed 150 -> 111 lines, "What's broken" no longer holds shipped
work, and the stale "After:" line (pointing at work that shipped on 2 August)
is fixed.
2026-08-03 11:36:16 +02:00
admin aa62449694 R-178 CLOSED: both demo boxes reinstalled from the merged golden and proven
gates / gates (push) Successful in 8s
Two boxes, two DIFFERENT supply paths, so the session proved the disk shape and
the delivery route rather than one of them twice.

demo-hp (layout proof, --golden <local volid>): mp0 at /var/lib/felhom,
backup=1, 70G, no mp1; /var/lib/docker and /mnt/sys_drive both real mounts of
its subdirectories via fstab; one df figure and one device id (64519) on all
three paths; reboots 3/3 with the binds surviving each.

demo-felhom (pipeline proof, --force-gitea-golden): fetch_verify succeeding
against the vouched manifest for BOTH artifacts -- 'verified sha256
54e2a4c431daf580... matches the hub manifest' for the golden, a7763d31... for
the agent. 250G single volume, grep -c '^mp1:' = 0, reboots 3/3.

Journey proven on both, endpoint-level: claim -> deploy -> back up -> restore,
with a planted marker returning byte-identical on each box. Ceiling measured
gone: 65 GiB and 233 GiB available to a recovery unit, against 19 and 45.

R-165 -> IMPLEMENTED, not PROVEN-LIVE, on the operator's ruling. B2, which that
row records as the bulkhead's replacement, fired live for the first time and
does refuse per app, delete nothing and alert -- but it is checked only in
captureAllRecoveryUnits while runVolumeDumps writes the bulk unguarded, and its
'the previous unit is untouched' claim was measured false (182,272 B dump
replaced by 2,147,666,432 B under a manifest still dated 06:34:26). -> R-181.

New: R-179 (uninstall leaves NAS network-storage units), R-180 (--archive-storage
not cross-checked against the ACL grant; 403 at step 8/8 after root@pam is
rotated), R-181. Third instance of R-115 recorded (agent 0.120.0 unpublished).

No code written, no version bumps -- this was a runbook.
2026-08-03 09:34:15 +02:00
admin 14d8c00781 docs: R-165 merge built and proven at the bake; R-163 + R-175 closed, R-178 filed
gates / gates (push) Successful in 8s
07-backup-architecture.md gains §7.5.1 (S-1: the contract changed in the same
session): the ceiling §7.5 describes no longer exists for a box built from
golden >= 0.192.0, the bulkhead's replacement is recorded, and R-175 is FIXED
here rather than left standing — the bound is restated as a function of mp1
and scoped to split-layout boxes, naming all three real shapes.

Capability map: new row as IMPLEMENTED, deliberately NOT proven-live, with
the missing leg named — no box has been reinstalled from the golden, and
"the golden baked" is not "a box built from it works".

R-163 CLOSED: the ceiling it recorded stops existing. R-176(a) answered by
P1; (b) WITHDRAWN, since every node is reinstalled rather than migrated.
R-178 filed for the reinstalls, which were not done this session.

CONTEXT S-13 (the variant chosen on measurement; pruning rejected with its
reason) and S-14 (prove first, then vouch — the golden is published but
deliberately unvouched, because vouching is what makes a fresh install pick
up a layout no box has been proven from).

STATUS: plain-language section; both operator questions now answered, so the
waiting-on-you item is cleared. Two older entries trimmed so the page did
not grow.
2026-08-03 07:16:15 +02:00
admin 41dbecb264 docs: R-167 + R-158 CLOSED, R-165 SPIKED, R-174..R-177 filed
gates / gates (push) Successful in 8s
R-167/R-158 shipped and proven live (controller v0.191.x, hub v0.89.0):
two new capability-map rows PROVEN-LIVE with live citations, and
07-backup-architecture.md §7.5's closing claim "nothing warns when an app
crosses the line" is now false and rewritten (S-1: an architectural
contract changed in the same session). §7.5 also gains the caveat that its
size bound is ONE BOX'S, not the fleet's.

Part 3 SPIKE (audits/SPIKE-r165-mp1-merge-2026-08-02.md): M1-M5 measured,
NO layout touched. Three findings the merge session must not re-derive:
"the layout" is not one thing (200G/50G vs 50G/20G vs 16G/8G); mp1 is a
BULKHEAD and not only a ceiling, so after the merge an overflow reaches
/var/lib/docker; the golden fails closed on the split in four places.
D-a's condition (1) is currently SATISFIED — no external box is in the
hub's register, and both demo boxes are Tier 0 and reinstallable.
Recommendation given, choice NOT made — it ends at the operator's ruling.

CONTEXT.md S-11 (D-c's routing, and why R-158's own backup_failed proposal
was overruled) and S-12 (the monitoring landed BEFORE the merge).
STATUS.md gains the plain-language section and the merge decision, with two
older entries trimmed so the page did not grow.

New rows R-174 (closed same session), R-175, R-176, R-177; each ID grepped
free before minting.
2026-08-02 23:56:16 +02:00
admin 8ef92a3fa7 docs: R-172 CLOSED (hub v0.88.0), R-173 filed, session report
gates / gates (push) Successful in 7s
R-172's root cause was not tuning — the WAL/busy_timeout pragmas had never been
applied, because the DSN used mattn/go-sqlite3 syntax against modernc.org/sqlite,
which ignores unknown parameters without an error. Recorded that way so nobody
re-reads it as "SQLite was slow".

R-173 NEW: while establishing who copies hub.db for the WAL change, found
pvc/hub-data labelled recurring-job-group.longhorn.io/default: disabled, with
backup-daily and backup-weekly the only recurring jobs and both on the default
group — so the hub database has no volume-level backup, and it holds every box's
break-glass root password plus the escrow custody records. Filed, not fixed:
whether the exclusion is deliberate is an operator question.

The session report is REPORT-r172-hub-wal.md, not REPORT.md, per the
parallel-session rule — REPORT.md belongs to the controller session that ran
immediately before this one.

It also records, plainly, that a 60-concurrent load test I ran OOM-killed the hub
pod three times against a 256Mi limit. Not the WAL change, and not a test I
should have run against a Tier-2 box; the unit tests already proved the property.
2026-08-02 21:22:14 +02:00
admin ad28699761 docs: R-157 A / R-170 / R-171 closed — boot recovery finished
gates / gates (push) Successful in 7s
Controller v0.190.0. Docs only here; no hub change, no hub version bump.

- audits/DIAG-bootrecon-drive-absent-2026-08-02.md — NEW. The Part 0 diagnosis,
  including the run that produced a FALSE NEGATIVE and the mechanism behind it
  (the agent re-binds an unmounted drive within ~60s, so the drive gate's startup
  reconcile restarted the apps one second before the sweep looked). Records that
  the write hazard was blocked only by an ACCIDENTAL filesystem permission that no
  code owns and no test pins.
- architecture/02 §0a — the boot-recovery contract (S-1): both gates read desired
  state; the sweep observes a SETTLED fleet and each sample must refresh first;
  nothing is started without asking, fail-safe. Plus the durable warning:
  Manager.StartStack has no gate of its own.
- 00-capability-map — the boot-recovery row, with the repeat count cited per N.5
  (6 of 6 hard resets) rather than a bare PROVEN-LIVE.
- OPEN-ITEMS / ROADMAP — R-157 CLOSED (both mechanisms), R-170 CLOSED, R-171 NEW
  and closed the same session, marked a regression from v0.189.0.
- STATUS.md — the power-cut line moved from "What's broken" to "What works right
  now" with its repeat count; one dated bullet in the change log.
- CONTEXT.md S-13 — the lessons worth carrying: "it didn't happen this time" is
  not a disproof; widening a window makes previously-unreachable overlaps
  reachable; and a settle detector is only as good as the freshness of what it
  samples — the fix's own defect, found live rather than by review.
2026-08-02 20:38:21 +02:00
admin 5c97fbc397 docs: R-166 SHIPPED — the desired/in-flight/observed split (D-b)
gates / gates (push) Successful in 8s
Controller v0.189.0 implements operator decision D-b. Docs only here; no hub
change and no hub version bump.

- architecture/02-controller-module-map.md §0a — NEW, and it is the S-1 contract:
  desired (app.yaml) / in-flight (own marker file) / observed (not persisted),
  with the rule that ties them — never derive one from another. Absent desired
  state means UNKNOWN, never "running". One file, one writer. D-b's binding
  safety rule quoted verbatim.
- 00-capability-map.md — the boot-recovery row now rests on a recorded signal,
  with the three live flows from 9201. The interrupted-operation half is marked
  IMPLEMENTED, not PROVEN-LIVE: nobody killed the controller mid-backup on metal.
- OPEN-ITEMS/ROADMAP — R-166 SHIPPED with both blocking facts and their answers;
  R-157 mechanism B CLOSED and A restated as the whole item; R-170 NEW (the
  drive-backed boot gate still infers a Stop from a container count).
- STATUS.md — the "an app can stay switched off and nothing says so" line
  rewritten to what is actually left: timing.
- CLAUDE.md — end-of-session checklist gains: confirm your own last push's CI run
  went green, BY RUN ID. The failure email is a push signal; this is the pull check.
- CONTEXT.md S-12 — the rulings, and the two lessons worth carrying: a test that
  constructs the thing it should prove the caller constructs is hollow (its
  red-proof will say so), and a field-by-field struct rebuild in a save path is a
  defect on sight.
2026-08-02 18:58:27 +02:00
admin c718aad1bc docs: R-168 SHIPPED, R-29 CLOSED on the demonstrated alarm, R-169 minted
gates / gates (push) Successful in 7s
SPIKE-ci-runner-2026-08-02.md: all six probes with method, measurement and ruling; none
STOPped. P2 (stock image has git but no python3) and P6 (a runner that loses its state
re-registers and orphans the old record) changed the design; P5 (a failed run signals
NOTHING) is why the alarm exists at all.

R-168 SHIPPED with its evidence. R-29 CLOSED — on the demonstrated alarm and not on a green
run, as required: the class it opened is answered at both ends, the hook refusing locally and
CI catching a --no-verify bypass and emailing. R-161 noted: its automatic half now exists for
the STATIC gate, while its original scope, the runtime gate, is deliberately still not
automatic and should stay that way.

NEW R-169 (grep established R-168 was the highest in use): CI can only report, because there
is no gate in the road. Making it blocking needs branch protection plus a PR workflow, both
of which change how the operator works — so it is theirs to decide, and the row states the
cost honestly rather than recommending it.

CONTEXT gains S-8 (CI detects, does not block, and why that is structural), S-9 (a detector
that tells no one is not finished, plus the curl and Cloudflare-1010 traps), S-10 (the runner
is unprivileged because DooPlex is Tier 2), S-11 (CI reproduces the sibling layout).

CLAUDE.md gains the rule earned by red-proofing: a go test -run pattern that matches no test
prints ok and exits 0, and an instrument that can silently drop results is not a measurement.
2026-08-02 16:35:34 +02:00
admin af2d103880 REPORT + STATUS: gate enforcement session, hub v0.87.0 live
REPORT overwritten per the standing rule; every red-proof, the core.hooksPath probe's four
measured outcomes, Scenario C's refusal-and-bypass, the hub deployment and the live Setup-tab
read are recorded there, plus three observations and two deliberate deviations from the spec
(a comment-only edit to felhom-host-install.sh, and __pycache__ in .gitignore).

STATUS: the 'check that needs a person to remember it' line is rewritten — the checks now run
themselves before every push, with both honest limits stated in plain words; and one entry
records the thirteen-check cleanup and the deleted installer version number.
2026-08-02 15:37:19 +02:00
admin e994bf35d2 STATUS.md: a plain-language operator page, and today's four decisions recorded
Documentation only — no code, no box, no build.

STATUS.md (repo root, 652 words / 67 lines): what works · what's broken ·
what we're working on · waiting on you · changed since. A VIEW of
OPEN-ITEMS.md, holding nothing of its own; not CONTEXT.md, and both files
now say why they stay separate. No R-n is the subject of a sentence —
identifiers are bracketed pointers only.

CONTEXT.md S-5 records the four operator decisions taken 2026-08-02
(D-a … D-d), none of them implemented:
  D-a merge mp1 into mp0 rather than resize it — before any external
      install, and D-c ships in the same step        → R-165
  D-b desired/observed app state in its own store, with the state-store
      safety rule verbatim                           → R-166 (BLOCKED)
  D-c customer fill warning + operator backup-failure alert → R-167
  D-d only DooPlex and Peti's box are protected      → target-selection.md

R-163 RE-FRAMED, not closed: the sizing question is withdrawn rather than
answered; the row survives as the record of the constraint until R-165
lands. R-156's papra referral RESOLVED — deployed nowhere, so the template
fix strands nothing; the docker ps evidence is recorded with its
provenance and its scope limit.

target-selection.md: two protected machines, everything else disposable.
ep0 is no longer Tier 2 but is not scratch (it holds the only off-premises
copy of real customer data) — flagged for explicit operator confirmation.
The demo-box backup-target fence drops from prohibition to stated cost,
because D-d spends that reference anyway.

CLAUDE.md gains an End-of-session checklist carrying the STATUS.md
maintenance rule and "a finding goes in OPEN-ITEMS.md first".
2026-08-02 14:20:29 +02:00