140 Commits

Author	SHA1	Message	Date
admin	c30430c530	skills: five process-domain skills + check_skills.py ... The four existing skills cover the product; nothing covered how work is reported. Two rules this project has paid for — check the artifact rather than the report, and do not state a claim more firmly than the evidence allows — lived only in the operator's head and in chat, where Claude Code never read them. - felhom-evidence five confidence tiers, artifact-over-report - felhom-diagnosis no hypothesis until a command has been seen red - felhom-plain-language ASD-STE100, two options, the re-pitch - felhom-handoff the note goes to a FILE, not the conversation - felhom-doc-authoring the pointer decides whether material is reached scripts/check_skills.py asserts what decides whether a skill is EVER reached: frontmatter parses, name == directory, description and body non-empty, under 150 lines, installed copy still samefile()s into the repo. install_skills.py globs and never reads the file, so a missing description installs perfectly and then silently never loads. It convicted on its first run: felhom-build-deploy is 179 lines. NOT trimmed here (pre-existing skills are out of scope, and trimming a deploy skill without exercising its commands is how a wrong command reaches a live host) — a named single-entry GRANDFATHERED exception, WARNed every run, R-394. A new skill over the limit is convicted. Red-proof run and seen failing: description removed from felhom-evidence -> exit 1, "frontmatter field 'description' is missing or empty". Restored, tree clean. skills/SOURCES.md records both MIT upstreams, that these are adaptations not copies, and the six pieces deliberately EXCLUDED with reasons. Register: R-392 (no architecture doc covers the two-AI workflow), R-393 (decision-log skill deferred, with the reason), R-394.	2026-08-25 09:36:20 +02:00
admin	ebdc04601d	docs(hub v0.108.0): the delivery grain, the cooldown ruling, and gate 11's first subject ... The alarm ladder gains §6.2 - which events are per-app, per-run, per-tier or coarse, and why the default is coarse. CONTEXT records two rulings: the grain is allow-listed rather than inferred from the payload, with crossdrive_failed as the proof that a payload rule would have been wrong; and a finding recorded only in REPORT.md has a lifetime of one session. R-389 closed and compressed, keeping its rules and naming the commit whose git show returns the full text. R-390 and R-391 left open. REPORT.md is gate 11's first real subject and passes: six observations, two FILED, four NOT-A-FINDING with their reasons. Three of those declarations are things a tidier report would have omitted - the gate's own spec would have passed the item it was built to catch, the burst has no ceiling, and ArgoCD said "successfully rolled out" while still running the old image. STATUS carries forward the one thing outstanding: the controller floor still reads 0.222.0 while the golden reads 0.223.0.	2026-08-23 14:12:23 +02:00
admin	2f7c9a6ce5	docs(R-329/R-386/R-387): the severity contract, the intent ruling, and Part 5 recorded ... The alarm ladder gains the severity contract (the hub's vocabulary is exact, it coerces silently, and three things now hold it) and the intent test with its three-way ruling on unknown. Both marked [DESIGN] with the live measurements. Part 5 is RECORDED AND NOT IMPLEMENTED: the operator's notification philosophy, verbatim, marked plainly as direction rather than current behaviour, with the 12 -> 15 toggle growth as the argument. Filed as R-388, a product decision. R-329 and R-386 compressed into CLOSED-ITEMS with their rules kept and the full-text commit named. R-387 filed closed - including WHY the dispatcher branch was kept rather than deleted, which is evidence (three monitor checkers call ProcessEvent directly) and not caution. The drill record names three things that had to be re-run: an inert red-proof mutation, Scenario G refused twice behind an HTTP 200, and the live Scenario A NOT proving the customer gate because demo-hp has no prefs row at all. Register: OPEN 328325 -> 328132 B, CLOSED 71441 -> 74642 B.	2026-08-23 12:03:49 +02:00
admin	ef6ac6fe74	One register, enforced by a gate; closed work compressed into siblings (R-376..R-378) ... Records and process only. No machine contacted. ONE REGISTER (operator ruling). 17 roadmap rows moved into OPEN-ITEMS.md keeping their identifiers, evidence and original filing dates - the oldest R-10, filed 2026-07-15, 38 days. 15 ideas stay in ROADMAP.md, which is their home; the gate exempts them by their own state word. 59 already-closed rows stay as history. Sorting rule recorded in the roadmap header: does the item assert something about the shipped product a reader could check and find false? scripts/one_register_gate.py, wired as the 11th gate. Control run: baseline passes, a planted open roadmap-only row is convicted by name, removing it passes with the file byte-identical, and a planted `idea` row is correctly exempt. Its four residual holes are in its docstring. The gate earned its keep immediately: it caught R-103, a READY finding my hand-sort mis-read as done because my regex matched the whole row where the body contains "shipped" - the gate matches the state cell. It also caught R-203 and R-163, recorded closed in the register and still open in the roadmap; the roadmap copies are marked SUPERSEDED with the register's verdict. HOUSEKEEPING. OPEN-ITEMS 672,376 -> 327,109 bytes (-51%); ROADMAP 239,306 -> 78,110 (-67%). Closed work compressed to 17% into CLOSED-ITEMS.md and ROADMAP-HISTORY.md; every entry names the commit whose git show returns the full original text. Rule-sentences are kept verbatim under "Reasoning kept" rather than judged entry by entry - 25 carry one. CONTEXT.md deliberately NOT compressed and the disagreement is argued in the report: 86% of it is standing rulings still in force, this prompt's own 3.4 says the log is never edited, and it has no per-ruling delimiter. Filed as R-377 - the problem is navigational, not volumetric. The hot/bulk placement decision was NEVER recorded as a decision anywhere - established, not assumed. Now marked [DESIGN] with a pointer honest about having no original date, given a decision-log entry that records what was rejected, and the [DESIGN]/[FACT] legend carried from 1 of 8 architecture documents to 8 of 8. Existing statements deliberately left unmarked (R-376). PROMPT-TEMPLATE gains N.7: compress what you closed, rehome live reasoning before it goes, state the register's size before and after. Ceiling R-375 -> R-378.	2026-08-22 12:13:54 +02:00
admin	091a4b7444	Correct the placement mis-framing, and file what we wrote down and never filed (R-368..R-375) ... Documentation and survey only. No code, no machine contacted. THE CORRECTION. The 40 catalogue templates without a configurable path are not missing a choice: 01-topology-and-trust.md:150-152 classes each volume hot (DB/config/cache -> fast storage, ENFORCED) or bulk (media/files), and the 40 are all-hot apps. The deploy page has been saying so to the customer all along (deploy.html:624-625). SPEC-app-data-placement and R-352 are corrected in place with the framing MARKED, not deleted; every measurement stands. R-356 was re-checked and survives, strengthened - an absent HDD_PATH is the normal state, so reading it as "not installed" misreads a correct configuration. The disk claim, precisely: since R-165 there is ONE guest data volume with two binds, not two volumes (build-golden.sh:29-40, 99). A physical-disk failure losing data and first-tier copy together is REAL and is what the other tiers exist for. A full data volume stopping the OS is NOT real and was the overstated one. THE SWEEP. 113 survey-class documents examined, 14 statements of "not filed", 2 already filed. Its positive control convicted the sweep itself twice before it convicted the corpus - markdown bold broke the strongest pattern, and the reporter re-searched a truncated line - both false zeros of the exact class being hunted, and together worth 2 of the 14. THE HEADLINE. The gap the 2026-08-21 drill rediscovered WAS filed - as R-107, ROADMAP.md:122, M/READY, 2026-07-28 - and is absent from OPEN-ITEMS.md, which calls itself the single source of truth. OPEN-ITEMS and that rule both landed 2026-07-27; R-107 went to ROADMAP alone the day after. 72 ids live only in ROADMAP, 29 not done, some of them findings. Filed as R-369 (HIGH). Five more still-open gaps filed with their ages: R-371 (17d), R-372 (38d, the oldest), R-373 (20d), R-374 (14d), R-375 (4d). R-368 corrects Part 4: the storage default IS applied at deploy time via deploy.html:612 - the earlier "the deploy route never reads it" came from grepping Go and never the templates. R-370 records the process failure and is closed by the template change. PROMPT-TEMPLATE gains the two rules it lacked: name the architecture document for the area and say what it says (with a file->area map and the test "is this something we chose?"), and an enumerated gap becomes a register row in the same session - a ROADMAP row alone does not count. Ceiling R-367 -> R-375.	2026-08-22 11:18:08 +02:00
admin	ab2262c91c	hub v0.106.0: report loss of visibility into the off-site stores (R-339) ... THE GAP, measured not supposed. On 2026-08-18 ep0's PBS proxy was wedged for 9 h 37 m and the hub emitted NOTHING on the operator channel. Both box checkers hold their last snapshot and return silently on a failed fetch -- correct for a FILL signal, since a missing reading must never be read as 0%, but it makes a dead off-site endpoint and a healthy one indistinguishable. The only mails that morning came from the boxes' own backup failures, and only because the WEEKLY offsite run happened to land inside the window. Two days earlier nothing would have fired at all. REACHABILITY is now a second, independent signal on both checkers: consecutive failed fetch windows, reported past a default 3 windows (~30-45 min) as pbsdr_box_unreachable / offsite_box_unreachable (warning) on the customer-less pbsdr-box / pool-box scopes, each with a paired *_recovered all-clear. Tunable via alerting.box_unreachable_windows (0/invalid -> 3). THE FILL LOGIC IS UNTOUCHED. No threshold, throttle, band or escalate-once behaviour changed; a degraded read still drives no transition. Three decisions a later reader would otherwise "fix" back, so each is argued in-code: - the unreachable event REPEATS rather than escalating once. The band shape would give exactly ONE mail at ~minute 30 of a nine-hour outage, and one mail is missable. It leans on the dispatcher's 1 h operator cooldown to become an hourly "still blind" heartbeat. - ErrUsageUnsupported is NOT blindness: an old ep0 answers "no such op", which means we reached it. Counting it would alert for days on a healthy pre-update endpoint. - born-blind is reported: the counter is not gated on having a snapshot, so a hub restarted INTO an outage still speaks. last_ok is OMITTED rather than zero-valued -- a fabricated timestamp reads as "it was fine until then". Both recoveries are severity "info" and severityNotifies drops "info", so they are registered in recoveredPairedDownTypes or the operator hears that the tier broke and never that it healed. A cross-package test drives ProcessEvent and asserts an actual operator MAIL, not a map entry -- a green checker test proves nothing about the seam (agent v0.91.0 shipped fully green with SetAuthSink never called). Tests: box_reachability_test.go (Scenarios A-F) + dispatcher_box_reachability _test.go (wiring). Three red-proofs run and reverted, each seen failing with a message naming the right cause: threshold 3->1, the sentinel counter guard, the pairing entry. Register: R-339 filed and marked SHIPPED (PROVEN-LIVE still owed -- no real or constructed outage has exercised the emit path, and one cannot be manufactured against Tier-2 ep0). R-340 filed: the reachability read rides ep0's LOCAL API daemon, which the incident explicitly cleared, so this check would have shown GREEN for all 9 h 37 m -- the honest boundary, recorded rather than glossed. R-336's next-step corrected: pvestatd's interval is NOT tunable (Proxmox staff have said so); the only lever is disabling the storage entry, which collides with the agent's consume-the-one-time-secret path. Doc-only, no agent code touched.	2026-08-18 19:27:33 +02:00
admin	0a5e9b14dc	due-checks gate (R-341), floor raise recorded (R-343), snapshot coverage (R-342) ... PART 1+2 — dated checks stop being wishes. R-341 booked two measurements as prose in a register row; nothing read those dates and nothing would have objected when they passed. The dates now live in a DUE-CHECKS block INSIDE OPEN-ITEMS.md (inside, so no sidecar can drift from it) and a new gate reads them. Registered as #10 in repo_gates.py, --fast, so it runs in BOTH the pre-push hook and CI. exit 0 nothing due (prints pending count + nearest date; empty block too) exit 1 a row is due/overdue (due <= today, UTC -- due TODAY counts), or a row names an item with no R-row exit 2 block absent/duplicated/unparseable -- INCONCLUSIVE, never 0 It REFUSES rather than warns, and its docstring states the limitation: it is NOT a scheduler, it fires on the next push, not on the date. 37 tests. BOTH red-proofs run and reverted -- and the first one earned its keep by catching a hollow assertion of MINE rather than confirming the gate: flipping <= to < left a due-today row in neither bucket, min() raised on an empty list, and the TRACEBACK exited 1, so "rc == 1" passed while the boundary was wrong. An exit code cannot tell a verdict from a crash. The test now asserts the conviction banner and the absence of a traceback, and the gate returns 2 rather than crashing if that partition breaks again. PART 3 — the floor raise, and the premise was WRONG. Read back from the store (not the form): min_controller_version = 0.216.0 @ 12:36:58Z, zero per-customer overrides, no "managed floor HELD" line. But read 5 shows the raise was NOT a no-op: demo-felhom had been on 0.214.0 since 12 Aug and auto-updated 0.214.0 -> 0.216.0 at 12:37:07Z -- NINE SECONDS after the save, exactly the immediate action publish-train rule 2 documents. No error events followed; it restarted clean. R-343 is therefore filed OPEN, not CLOSED: the closing condition was all five reads clean and no directive served. It went well, but a record calling it inert when it moved a customer box is what misleads the next reader. The row also states why the floor was behind -- rule 2 policy, not drift, earned by the 2026-07-11 skew onto Peti's box -- and cites ResolveManagedFloor (store.go:2068) plus the two build-felhom-iso.sh facts (build-time at :267, fails open at :78-82) rather than asserting them. Two boxes are below the floor and neither reports: drill-r50 (blocked, powered off) and peti-felhom (host row deleted). peti-felhom was NOT contacted -- its row records that a report from a deleted host 401s and is not persisted, so the raise cannot reach it. PART 4 — R-342 filed READY, quoting stop2-snapshot.txt verbatim: Hetzner server snapshot 421440873 covers /dev/sda only; /mnt/pbs-datastore is a separate Volume that snapshots exclude, so a rollback restores software state and NOT the datastore. Fine for that upgrade; the safeguard for any future procedure that could touch the datastore does not exist and is Viktor's call. Also: CLAUDE.md's gate list named 6 of 10 registered gates -- completed rather than appending a 7th to a wrong list (124 -> 128 effective, ceiling 200). Capability map deliberately unchanged; no row cites a floor or golden version. repo_gates.py fully green, 10/10.	2026-08-18 15:16:51 +02:00
admin	e0b56c976f	REPORT + CONTEXT: the third name, the second door, and a number that answered a different question ... Three rules carried forward. A name must separate on the STEM, not the noun — naming this secret after the act it is used in would have recreated the trap, because the other factor on the same page is the „Párosító kód". A guard is worth what its positive control is worth: this one's selftest convicted its own step-3 case and found a defect in the guard itself. And a suppression must rest on the machine's own declaration, then be checked for the SECOND door — recording the disabled state rather than deleting it is what let the deadline check skip it too. Yesterday's report is preserved to audits/ because it carries the only record of the self-heal verdict (Part C was dropped, so that reasoning is in no register row) — the rule written last night, applied to itself the first time it mattered.	2026-08-13 16:01:45 +02:00
admin	955a4f07b7	REPORT + CONTEXT: A4 reverses its own premise, the naming half, and the first R-264 reader ... A4's evidence contradicts the task's framing and the register now says so: peti-felhom is a real machine with 482 reports and a real person behind it; david -> tester-1 is a record that has never had a host, an escrow or a report. The risk is real; only the word that named it was wrong. CONTEXT gains three standing rulings: read the fact that carries the RISK (heals_last_hour, not state) and decode the one that decides the question (heal_succeeded — R-260's lesson); one secret in two situations keeps its name and changes its sentence, and the mail must name the page the machine actually shows; and evidence dies in the INTERMEDIATE revert — with the corollary that a durable citation may never point at a file whose contract is to be overwritten, which is why the R-316 report was moved to audits/ before this one was written.	2026-08-13 10:59:55 +02:00
admin	fc737b0fc0	installer v1.28.0: the removal genuinely reverses the installation (R-316) ... v1.27.0's fix worked exactly once per machine. Measured on drill-r50 from virgin, on the PUBLISHED v1.27.0, before anything was changed: cycle 1 recorded 'no' and freed :53; cycle 2 recorded 'yes' and left dnsmasq running on 0.0.0.0:53; cycle 3 refused, exit 1. Every box already in the field is at cycle 2, and a reinstall onto a machine that has had Felhom is cycle 2 by definition. Why cycle 2 says yes: the preflight's ownership question is dpkg-query package presence and nothing else - not the absence of a record. Stopping the unit and leaving the package made our own package read as the household's one cycle later. Now the uninstall removes the package when the record says we installed it. Order unchanged and load-bearing: read the record, act, then delete the state file that holds it. TWO packages are recorded, because dnsmasq ships the unit and dnsmasq-base ships /usr/sbin/dnsmasq, and each is taken back only if we added it. The dependency check is a SIMULATION, not a guess: apt-get -s purge is asked what it would remove and the purge proceeds only if that set is a subset of ours; otherwise stop+disable, naming the package that blocked it. Never interactive, never fatal, and the success is re-queried rather than read off an exit code. Watched: three fixed cycles -> install 3 PASSES; a household resolver untouched; a dependent package not purged and named; no record -> untouched with the command named. Red-proofs with the mutation asserted applied: remove the purge -> cycle 3 refuses in those exact words; remove the ownership check -> a household resolver is purged; infer ownership -> the guess is taken. Also: R-317 (the agent stats a path dnsmasq-base owns to decide whether to install dnsmasq - pre-existing, now reachable), R-318 (no honest ownership marker exists for existing boxes; the preflight message is the mechanism), and the status page's decisions section rewritten to say what each decision costs and what doing nothing selects.	2026-08-13 08:16:04 +02:00
admin	8b188bea68	hub v0.103.0 — a host can read the packages we kept for it (R-311) ... ListSupersededEscrow had zero production callers for nineteen days. It is the only reader of a retained identity_blob, so the retention shipped in v0.93.0 was material the product could not reach - proven on the fixture 2026-08-12, where a code that opens a retained package was answered as a code that opened nothing. New GET /api/v1/hosts/<id>/escrow/retained: self-scoped exactly as the current-row GET, same recovery-mode gate, same audit event written BEFORE the bytes leave, capped at 16. Rows with a NULL identity_blob are WITHHELD and returned as unopenable_count - they retain the PBS key, not the repository password, so they can never open what the caller is asking about, and serving them would let the screen claim an earlier package is openable on exactly the boxes the original defect hurt. The count is returned because their existence is load-bearing and underivable by the caller. The trade, stated rather than waved through: the hub still cannot read any of it - sealed bytes in, sealed bytes out, no decrypt path, no recovery code ever held. What widens is volume, bounded by self-scope, the recovery-mode gate and the cap. The response is a NAMED TYPE, not a map, so the wire-contract gate can resolve it; the wire is declared as a fourth ROOT and the gate now checks 182 tags rather than 174. A positive control shows that check is name-presence, not decodability - filed as R-315 rather than reported as coverage. Also: golden 0.214.0 baked, published and round-trip verified; the countdown on demo-felhom cancelled on the operator's ruling (R-307); the spike that halted Part 3 recorded as R-312; the set-aside store found unrecoverable as R-313. Six hub tests through the real endpoint; four red-proofs asserted applied.	2026-08-12 18:56:44 +02:00
admin	fbe1155fbb	R-302 docs: register rows, the two rules earned twice, STATUS ... Closes R-296 (verified: shipped in v0.212.0) and R-301 (premise confirmed, fixed in v0.213.0). Files R-302 with WHY the obvious condition was rejected, and R-303 for the missing markOrphaned guard - the co-render is now harmless, not impossible. Bake evidence for golden 0.213.0.	2026-08-12 15:38:11 +02:00
admin	c04f933d0b	The census answers no, three receipts found, and the prune was on file all along ... CENSUS (read-only, hub store, tester's machine not contacted): no machine that is not ours can be in the state that cost demo-felhom its history. The hub holds escrow for three hosts; both demo boxes lost their pre-fix key in the same four hours on 2026-08-04; peti-felhom and david have no host row and no escrow at all. A control ran FIRST and had to pass -- the query returned "present (572 bytes)" for a host known to have material and "absent (NULL)" for one known not to. Corrected my own instrument on the way: a date-only comparison mislabelled both losses as after the fix, so the in-force moment is now pinned from the hub's first post-fix escrow row (11:11:37Z), which independently agrees with the register. PART 1 ESTABLISHED. The prune is recorded inside R-267 -- the row about the Configuration page being slow -- because pruning artifacts is what made that page fast. Arithmetic checks (23+7=30, plus three versions that only surfaced after the first thirty moved them onto page one = 33) and the PAGINATED listing shows both generics at exactly ten. R-291's blocking condition is released: the operator was being asked to establish something already written down. And my counter-argument yesterday was wrong in exactly the way R-267 warns about -- "containers hold 19" came from an unpaginated query; paginated they hold 270 and 169. RECEIPTS: three restored (drives.enrol, backup.tier1, fail.lost-recovery-code), each citing the document that walked it; the map already read PROVEN-LIVE for all three, so this follows the map rather than raising a status in the view. NINE HONEST GREYS. fault.selfheal's best hit argues against it -- an incident recording self-heal's absence through a 1h15m outage. THE DECAY RULE FIRED FOR THE FIRST TIME. backup.restore-proof has a receipt from 28 July and is superseded anyway: demo-hp's restore-test failed 5 August and the box has since been rebuilt. A claim about a continuing behaviour cannot rest on an old observation. The capability map still reads PROVEN-LIVE and is now the thing out of step -- recorded, not silently rewritten. PART 4 specified, not implemented. The orphan card promises restorability the box rendering it cannot evaluate: the discriminator is on the hub and no wire field carries it. A conditional promise the system cannot evaluate is the same defect as an unconditional false one, so the copy stops promising, says what happens, and names a route. Ships with the next controller change so one bake covers both.	2026-08-10 11:15:37 +02:00
admin	985f0ba63c	Record the guards, the narrowing, and the two things I could not do ... R-273's owed guards are both built and closed. R-291 records what CI stopped covering and why, so it can be widened deliberately rather than discovered. R-292 is new and was found by a test failing for the wrong reason: artifact_sha_invalid conflates "version missing", "registry unreachable" and "bad sha" into one message. v0.102.0 works around it by ORDERING -- the probes run first, so an unreachable registry is reported as unreachable -- but the message itself is untouched. CONTEXT gains the rule this session is about: a check and the policy it enforces must read the same number from the same place, or they drift and the drift looks like a defect in something else. Two corollaries, both of which cost something: a bounded check must print what it stopped covering on every run, and an unreadable policy is INCONCLUSIVE rather than unbounded. Stated in the report rather than glossed: Part 4 (finding receipts for the twelve downgraded claims) was NOT done and is a shortfall, not a decision -- splitting it would have produced exactly the half-checked green the exercise exists to prevent. Part 5 was droppable and dropped. The tag-push green is not re-proved tonight and is not claimed; the evidence offered is runs 190 and 216.	2026-08-09 19:28:45 +02:00
admin	4a4a1e245a	R-265 CI timeout + golden 0.210.0 baked; R-221/R-259/R-258 closed, R-266 minted, G-3 unblocked ... Four defects of one family, all shipped today: something the box already knows, thrown away or drawn as its opposite. Agent v0.128.0, controller v0.210.0. NO HUB CODE, no hub bump, no ArgoCD sync. R-265 (this repo). timeout-minutes: 5 on the gates job — every honest run in the observed session finished in 18-34s, so this is ~9x the slowest and far under whatever reaped run 264 at 834s with no log. The alarm mail now carries Elapsed (start stamp via $GITHUB_ENV; an absent stamp prints "unknown (no start stamp)", never a bogus 1.7-billion-second figure) and its "names itself in the run log" sentence is qualified so it cannot mislead when there is no log. ⚠ THE UNKNOWN IS NOT CLOSED. Whether the if: failure() alarm fires for a REAPED job is still unverified. The timeout makes the reap unreachable in practice; it does not answer what happens in one. Demonstrating it means deliberately hanging a run on main, which would leave the branch red for a parallel session. Said in the workflow comment, the changelog, R-265 and the report — none of them claiming it is answered. GOLDEN 0.210.0 baked, published, round-trip verified, NOT VOUCHED. The currency gate went red the moment the controller was bumped — correct — and is closed by the bake, never --no-verify. No --no-verify anywhere this session. ⚠ THE AGENT WAS NOT PUBLISHED UNTIL THIS SESSION CHECKED, AND IT MATTERED. R-221's fix is in the AGENT, and a fresh install takes its agent from the Day-0 manifest. The binary had been hand-deployed to felhom-pve and never published, so agent_version 0.128.0 was not selectable and a fresh install would have received 0.127.0 — the golden would have carried the controller fixes and NOT the one the headline defect needed. Caught by checking each Day-0 value was FETCHABLE rather than assuming. Published from the live-deployed bytes, sha-verified across the hop first. Registers. R-221, R-259, R-258, R-265 CLOSED. R-266 MINTED (READY): the failed root statfs still travels to the hub as a 0-of-0 disk; ranked LOW because it is the quiet direction — it can only miss a true alarm, never raise a false one — and it is now a two-repo wire change governed by G-1's gate. Highest ID moved R-265 -> R-266. CONTEXT S-39 rules the convention this project was missing: "we do not know" is never drawn as "fine", and the codebase has ONE way of saying it — an explicit ...Known bool companion checked in the template. ROADMAP G-3 was explicitly blocked on that decision and is unblocked; what remains there is a survey-and-convert of existing sites, not the gate. Capability map row 93 CHECKED and it was NOT claiming something untrue — it is about the operator notification path. But its narrative ("the page you open to ask whether ONE app is backed up") invites the wrong reading, and the adjacent thing WAS false until v0.210.0, so the row now records that the two halves disagreed and only the operator half was true. Six red-proofs across the two code repos, each with the mutation asserted applied. The one that matters: Part 1 Scenario A FAILED against today's tree, with the intended message. Part 1's operator-present live validation is OWED and is the session's STOP. repo_gates --fast: all 8 OK.	2026-08-08 16:52:39 +02:00
admin	650cc8a4a7	docs: record the CI episode — three instrument defects, and the two-homes rule (R-260) ... The gate's third instrument defect and the reusable lesson underneath it: a gate needing a sibling clone passes on a workstation and is INCONCLUSIVE in CI, and a gate must not shell out to a tool the CI image may not have. Three red runs (260-262), each of which mailed the operator, before 263 went green. CONTEXT S-38 gains the two-homes rule.	2026-08-08 09:12:29 +02:00
admin	b080ecf411	hub v0.99.0 — the hub can see whether the operator can get in (R-260); G-1 gate closes, R-247 closes ... oobDegraded tested five things and the sixth never arrived. The agent has emitted `operator_key_configured` on every heartbeat since v0.72.0 — the SAME version that introduced the `oob` stanza carrying it — and store.HostOOBRow mirrored five of the agent's eight OOB fields. With no field for it, encoding/json discarded the fact on arrival, so a box with felhom-sshd active, reachable, a valid config and a configured peer reported `ok` with NO OPERATOR KEY INSTALLED AT ALL. Not a wrong answer: an answer to a question nobody was asking. `operator_peer_configured`, which the hub did read, only says the peer IP is in desired-state — that OOB is MEANT to work, not that entry is possible. Now decoded: operator_key_configured, plus wg_handshake_age_s and healed_at. The last two ride the ALERT TEXT and are deliberately NOT in the predicate — widening a check beyond the fact that is now arriving is how a check stops being read. SCENARIO F, decided on a measurement rather than a preference. operator_key_configured decodes as a POINTER: nil = the agent never said, reported distinctly and never as ok. The version gate was rejected because the field and its stanza shipped in the SAME agent version (v0.72.0), so a stanza without the field cannot come from any released agent; the fleet is 0.113.0/0.127.0 and the vouched floor is 0.127.0. Handled explicitly anyway and pinned, because "cannot happen" is a claim this project has been burned by. THE MESSAGE NAMES THE FAULT. oobDegradedReason is the single source for both predicate and text, so the alert can never name a different fault from the one that fired. The old form derived it separately and had a vocabulary of two — unreachable, or config invalid — with no way to say the key is missing. The operator reads this at 07:00. TESTS DRIVE THE DECODE BOUNDARY. Every hub OOB test before this built a HostOOBRow by hand, and a test written that way CANNOT SEE A FIELD THAT NEVER DECODES — which is how this held a green suite for five weeks. The pre-existing fixture oobReport() also omitted the field, so those scenarios ran against a report shape no released agent produces (same family as R-262). Both fixed. Red-proofs, 8 expected outcomes and 0 wrong, each with the mutation asserted applied: dropping the field returns the false ok; an unconditional check alerts a healthy box; unknown-as-ok restores the silent pass. G-1 CLOSED — scripts/wire_contract_gate.py shipped as ranked, built BEFORE the fixes and seen failing on 40 fields (documentation/tests/wire-contract-gate-2026-08-08/BEFORE.md). Two instrument defects the control caught first: a substring false negative (grep -F healed_at matched privsep_healed_at) and treating dr_recipe as wholly opaque when its top-level sections ARE decoded through an allow-list that already cost offsite_restic (R-122). The prompt for this session said "465 emitted tags, eight unreachable". Checked against the repo: R-260 said "at least eight DECISION-BEARING facts", never eight tags. The real count is 40. R-260 CLOSED (class gated, sharpest instance fixed). R-247 CLOSED (controller v0.209.0). R-264 MINTED and OPEN — the 21 facts with no consumer, allowlisted with reasons so that gating the class could not be mistaken for deciding them. Still open and named: R-246, R-255..R-259, R-261..R-263, and C7's test-comment half. Capability map checked: it claims OOB access is implemented, never monitored, so no row was untrue; what was untrue sat one layer down and the row now records it. repo_gates --fast: all 8 OK. go build/vet/test green in hub, run separately from this commit.	2026-08-08 08:47:02 +02:00
admin	a1a542b9a7	RECON: what DooPlex's own backup actually protects (R-232) — read-only, nothing changed ... Five sets, 14/14 successful runs in 14 days, and a file restored byte-identical from the data repo -- the first time a restore has ever been demonstrated here. What the failure matrix exposes: notify_failure is a no-op (NOTIFY_ON_FAILURE=true, webhook commented out) so a failed backup tells nobody; NOTHING leaves the box (Longhorn's target is DooPlex itself over NFS, the only outbound-looking cron pulls inbound for another project); the backup tree is a single writable path with no append-only; two sets are same-disk (.claude-memory and the postgres dumps, whose source sits inside the backup tree); Longhorn retains 1 generation; secrets/restic-repo has never held a snapshot because backup-secrets.sh never calls restic; and /opt/backup/docs/BACKUP-RESTORE.md does not exist though the unit advertises it. NOT a finding: the restic passphrase. On-box copy is on a different disk from the backups and the operator holds an offline copy out of band, so disk loss is recoverable. The residual is only that it is operator-held rather than system-held escrow. A stale lock in the data repo is MINE, from the previous task's verification. Fenced from clearing it; recorded with its impact and the one command that fixes it.	2026-08-06 12:16:38 +02:00
admin	5ca5082e7c	docs: close out the instruction arc — t740 corrected on evidence, registers, ledger, S-37 ... target-selection.md said demo-hp has no off-site tier. Measured first: pvesm list felhom-pbs on the box returns two snapshots in demo-hp's OWN namespace (2026-07-28, 2026-08-04) against ep0's felhom-offsite. The claim was TRUE WHEN WRITTEN and went stale when F10 resolved 2026-07-23. The measurement is kept in an HTML comment beside the corrected sentence. This file decides which machine may be destroyed, so the sentence was load-bearing, not cosmetic. R-229(b) CLOSED (agent 175 -> 99 eff). R-230(b) CLOSED (symlink, proven from fresh sessions). R-230(a) part-actioned -- three false statements fixed, WARN loop added, bulk ruling still owed. S-37: a claim in an instruction file is checked, not trusted.	2026-08-06 11:49:22 +02:00
admin	9c10c5d600	docs: ledger, registers and standing ruling S-36 for the part-2 instruction/memory work (R-229) ... R-229 legs (a) and (c) CLOSED. R-230 opened for what this deliberately left -- the ruling owed on auto-written staleness, the symlink decision, the spec-as-failing-test pilot. R-231 opened for /opt/backup/scripts/ being unversioned host state, found while adding the memory store to the backup set: the same class this task was closing, inside the fix for it. S-36: the auto-memory store is backed up, never committed; no memory file is ever deleted; a rule file created mid-session is NOT live in that session. Second session in this repo, so REPORT-<topic>.md per CLAUDE.md; shared REPORT.md untouched.	2026-08-06 11:01:13 +02:00
admin	c21bcf84f7	docs+gate: instruction files cannot silently regrow (R-229) ... New shared scripts/instructions_gate.py, registered in controller_gates.py and agent_gates.py, never copied into a sibling repo (the reuse_refs_check.py precedent). 20 fixture tests, all asserting the effect: exit code AND that the message names the file and the reason. It is a consistency gate, not a budget gate, and the failure message says so. A /context reading measured the instruction files at 15k tokens against 869k free in a 1M window -- space is not the constraint, and a future reader must not re-derive the wrong reason. The 200-line ceiling is adherence guidance; a file nobody can hold in their head is where contradictions hide, and five were found here. Checks run against effective text (HTML comments stripped, because they are stripped before injection): the line ceiling; every .claude/rules/*.md declares paths: or an explicit unconditional: true; no component version literal; no TEMPORARY block carrying a past date; and the workspace-root CLAUDE.md is byte-identical to its versioned copy -- the live file sits outside any git repo, so that copy is its only version-controlled record. Two traps recorded so they are not reintroduced: a bare \d+\.\d+\.\d+ matches the first three octets of every IPv4 (the gate excludes dotted quads, or it fails on 192.168.0.180 in the agent's own file); and unconditional: true is NOT a Claude Code feature but this project's own marker. Workspace-root CLAUDE.md 208 -> 182 lines (142 effective), copy kept identical. The nine-instance invariant table moved into the felhom-testing skill, which triggers when writing or reviewing a test; all three directive bullets stayed in the core. felhom.eu/CLAUDE.md got surgical corrections only and is knowingly still over the ceiling at 227 effective lines -- closing it needs the restructure R-229 defers, said plainly rather than quietly absorbed. CONTEXT.md gains standing ruling S-35. OPEN-ITEMS.md gains R-229. Docs only -- no Go, no version bump, nothing built or deployed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JJc8sAGRWmavP3rMtdpkr2	2026-08-06 09:38:52 +02:00
admin	f45b1f6761	docs: R-193 CLOSED (the recovery screen); R-213 minted for the put-back ... - OPEN-ITEMS: R-193 CLOSED with both 2026-08-05 rulings (unlocking and restoring are separate; 'I do not want the old data' moves the store aside after a double confirmation), and the shape-(b) reasoning — WriteOffboxSecrets auto-generates a repository password on re-apply, so the literal 'fresh data area' trigger would have opened a window that closes by itself. - R-213 MINTED (R-212 was and still is the highest, re-checked for the second writer): putting files back in place, with the live-versus-backup comparison named as its requirement. Not started, deliberately. - capability map: the 'needs someone who knows to look' qualifier is GONE; what remains is stated narrowly — no correct-code run through the page, the put-back is out of scope, and the journey has not been re-walked end to end. - 07-backup-architecture 7.0: a fifth row, and where the screen deliberately stops. - CONTEXT: standing ruling S-34. - STATUS: the headline change and the two things still owed as proof. No hub change and no hub bump.	2026-08-05 12:56:55 +02:00
admin	4faebe2926	docs: R-204 ALL FOUR items closed; R-193 credential half; R-192 by replacement; R-212 filed ... - OPEN-ITEMS: R-204 all four CLOSED with both 2026-08-05 rulings recorded (the declared-state trigger and its four-meanings-of-absence reasoning; the recovery preview's dashboard-password exposure accepted as metadata, not content). R-193's credential half CLOSED, screen + deletion still open. R-192 CLOSED by REPLACEMENT. R-202 untouched. - R-212 MINTED (R-211 was the highest, grepped): the orphaned-ciphertext deletion HALTED at its STOP because the measured paths do not match the register — three set-aside stores totalling ~1.45 GB, and the thing that is exactly 1.2 GB is demo-felhom's LIVE repo. Nothing was deleted. - capability map: all four interventions closed; the row KEEPS a qualifier for a new reason — no step needs an operator, but there is no customer-facing recovery screen, and the journey has not been re-walked end to end. - 07-backup-architecture 7.0: the four-step table updated; the declaration-vs- inference reasoning and the credential-automatic/key-customer-present split. - CONTEXT: standing ruling S-33. - STATUS: the headline change and the deletion STOP. - REPORT-r204-item4.md rather than REPORT.md: a parallel session is active in this shared clone.	2026-08-05 11:06:41 +02:00
admin	0dbd954fec	docs: R-196 closed, R-204 items 1-3 closed, item 4 open (R-193) ... - OPEN-ITEMS: R-196 CLOSED; R-204 items 1-3 CLOSED with item 4 named and its dependency stated. Header restates that R-202, the 1.2 GB ciphertext deletion and R-198's still-unit-proven retention all REMAIN OPEN. - capability map: the recovery row keeps its 'with a person present' qualifier, names which crutch remains, and cites the three now gone. - 07-backup-architecture: new 7.0 - what a customer can and cannot do ALONE, the four steps in a table with status. This is the section a future reader will use to answer that question. - CONTEXT: standing ruling S-32, superseding S-31 steps 2-5. - STATUS: rewritten to one screen per its own header; removes a corrupted half-overwritten section left from the drill session. - ROADMAP: R-196 and R-204 collapsed.	2026-08-05 07:37:35 +02:00
admin	2a7ac03c47	R-201 PASSED: a customer's file survived a machine rebuild and came back byte-identical	2026-08-04 23:18:47 +02:00
admin	b228fd102d	R-201 night run: the off-site key IS recoverable after a real rebuild (proven); the verdict is blocked by R-204	2026-08-04 22:33:01 +02:00
admin	a0c4b607a6	drill R-201: prepared and HALTED before the wipe — a mandatory data directory was missing from the off-site snapshot while the run said ok (R-203)	2026-08-04 15:00:06 +02:00
admin	846253baa8	docs: the offsite key is proven recoverable (R-199 closed, R-200 half, R-201 scoped)	2026-08-04 13:55:25 +02:00
admin	91cabdde1b	hub v0.93.0: the retention keeps the key it was built to keep (R-198) + three honesty fixes (R-197, R-192, R-196) ... R-198 — host_escrow_superseded shipped with `blob` (the K-escrow / PBS datastore key) and identity_blob was added to host_escrow LATER, never here. The offsite restic REPOSITORY password lives in identity_blob. So demoteCurrentEscrowTx -- whose own comment calls it "THE ONE escrow row-copy routine" -- retained the whole-guest key and silently dropped the off-site data key, which is the secret the retention was built to preserve. And because the copy happens as the new blob overwrites the old, the destroying act was the ESCROW CEREMONY: the exact thing a rebuilt box tells its customer to run, on a card promising in Hungarian that the old backups stay recoverable. Both demo boxes crossed that line on 2026-08-04. - identity_blob added to the table (CREATE + additive ALTER) and carried in the shared copy routine, so BOTH callers are fixed at once: re-escrow and host-delete demotion. - ListSupersededEscrow reads it back; store.HostEscrow gains IdentityBlob. - CountCurrentEscrowWithIdentity is the census of who the fix protects. - Nothing is backfillable: pre-v0.93.0 retained rows have no blob and their sources are gone. - Tests assert the CONSEQUENCE (a retained row can still yield a repo password), which is why the pre-existing retention test stayed green for two months asserting the mechanism. R-197 — SaveHostEscrow returns the hash it replaced; the escrow PUT raises offsite_repo_key_changed (warning, operator-only, edge-triggered) when both hashes are known and differ. No hash value travels. Severity chosen for the world v0.93.0 creates: with the identity blob retained, a changed key is "this history now depends on an older recovery code", not a loss. R-192 (half) — the stuck alert now reports the two shapes it actually covers, burned and regressed, each stating its own measurement; the regressed text withdraws the Re-issue recommendation. Every self-heal refusal leaves a notification_log row with its reason. The guard's logic is unchanged; its 500-oldest-reports scoping stays OPEN and the window is named in the alert text so the limitation travels with the number. offsite_delivery_stuck and offsite_credential_restaged are added to operatorOnlyEvents -- neither was registered and neither has a customerMessages entry, which is not a block. R-196 — five comments (not the three the spec expected) claimed ReissueCredentials rotates the restic repo password. It resets the PROVIDER password and cannot touch the repo password, which is generated on the box. All five corrected; the staleness mark documented as precautionary. The BEHAVIOUR stays open. Not in this release: R-199, R-200, R-201 remain open -- the chain that hands the key back is still unassembled. Part 5 hit its gate; the orphan card is untouched (R-202).	2026-08-04 12:56:58 +02:00
admin	7fff45d688	R-195: a customer with no machine ever bound does not alarm (hub v0.92.0) + R-193/R-192 spike ... Part 4 (ships): `david` — a prospective customer with hosts=0, host_deletions=0, reports=0 — e-mailed an expected_dbdump_missed ERROR at 03:00 UTC three mornings running. The existing down-skip could never cover it: it reads the staleness checker's state, which is seeded from a query over the `reports` table, so a customer that never reported has no state at all and GetState() returns "" rather than "down". store.HasEverBoundHost (hosts row OR host_deletions tombstone) is consulted once per customer at the top of the deadline loop. The discriminator is "was a host EVER bound", never "has a report arrived" — a box installed and never heard from is a real fault and keeps alarming. Fail-OPEN on a read error. Red-proof observed: removing the guard fails with `got [expected_dbdump_missed]`, verbatim the event david sent. Parts 0-3 (spike, NO production code for R-193/R-192): audits/SPIKE-offsite-credential-recovery-2026-08-04.md establishes that the one-shot provider password is the RECOVERABLE secret and the restic repository password is the irreplaceable one — and that a guest rebuild mints a fresh one, orphaning the previous off-site history. Measured without touching a box, by comparing host_escrow.restic_pw_sha256 against host_escrow_superseded: BOTH demo boxes changed (demo-hp 15 snapshots / 40.9 MB, demo-felhom 36 snapshots / 1.14 GB). demo-felhom's "lucky" 76-second recovery restored delivery and not the repository, silently, for 13h. ReissueCredentials does NOT rotate the restic password (R-39's record and two hub comments are wrong -> R-196); candidate (b) is not implementable against a zero-knowledge escrow; candidate (a) already exists as F3 and is wired to the wrong event. Ends in ranked options and an unanswered question for the operator. R-195 SHIPPED; R-196 + R-197 filed; R-192 + R-193 updated, neither closed.	2026-08-04 11:04:39 +02:00
admin	a684a81796	docs: R-191 closed, R-190 mitigation shipped (mechanism open), R-194 filed ... R-191: the gate was run first and it mattered — ep0's prune jobs have run daily since 2026-07-27 (18 tasks, all OK). THREE of my own queries said the opposite and all three were broken instruments (worker-type vs worker_type, prune vs prunejob, proxmox-backup vs proxmox-backup-proxy); acting on any would have disabled the only pruning attempt while reporting that nothing prunes. R-190: the row now reflects the MITIGATION, not the cause, and says so. Includes that v0.124.0's transition record failed in production while every unit test passed, and Part 3's single pass — the token-rotation class is real and documented in our own installer but does not fit this box (same token still authenticates, the other three grants survived, no installer run evidenced); ruled out on this box, not fleet-wide. R-194: PVE caches permissions (~40s and ~16min measured), so a single permission read is a lagging indicator and may explain when R-190's refusal surfaced. 07-backup-architecture: the offsite retention row was RIGHT and the configuration disagreed with it — noted, since a doc that states a contract does not enforce it.	2026-08-04 10:04:25 +02:00
admin	d692c46db0	RUNBOOK: the host tier is restore-PROVEN unattended on both demo boxes ... Four SCHEDULED runs, none triggered by hand: demo-felhom host 83.8s / offsite 540.4s; demo-hp host 109.3s / offsite 300.1s. Each restored into a scratch guest, booted, verified and destroyed itself; zero 990000 guests or volumes afterwards and both local-lvm figures returned to their pre-run values. Both boxes had BOTH tiers due at once, so R-86's ordering was observed live for the first time: never-proven sorted first, each box took its HOST tier, deferred the offsite one, and picked it up on the next evaluation six hours later. The host-tier proofs reached the hub through R-189's merge — demo-felhom's report carries two tiers, and the local one can only have come from disk. The capability map's optimistic half is cashed, with its scope stated: these two boxes, not the fleet. Surfaced and filed rather than fixed: - R-190: a storage ACL that demonstrably worked at 04:44 was gone by 09:24, with a reinstall, any logged pveum activity and any cluster-log entry ruled out. - R-191: every weekly offsite backup uploads successfully and then fails the job on a prune the box is deliberately not allowed to do (R-89 moved it server-side; both boxes still arm keep_last=2). Two corrections to yesterday's record: the R-185 drift DID surface as 403s on the write path (six, with the hub raising whole_guest_backup_failed at the first), and my earlier "no restore_test_* events" was produced by grepping a 404 page.	2026-08-04 08:13:43 +02:00
admin	e3187c86d5	docs: R-185 closed — the silence as well as the grant ... - OPEN-ITEMS: R-185 closed with the measurement, the corrected root cause (the installer's Scenario-F reuse arm, not PVE_STORAGES), and the live sequence. Records that demo-hp carried the same drift and was fixed too. - capability map: the whole-guest row's HOST-tier half was OPTIMISTIC and now says so — that tier was not merely unproven, it was unprovable on both demo boxes, and every live proof cited was on the offsite tier. - vzdump-target-move runbook: its item 5 predicted this; annotated (not rewritten) with what actually happened — the create arm did grant, the reuse arm did not, and it surfaced as a silent unreadable tier rather than the 403 the item expected, because vzdump writes through a root path. - CONTEXT: S-21 (an empty listing cannot distinguish forbidden from newborn; the measured trap that an ungranted path answers with INHERITED privileges) and S-22 (the Scenario-F arm must finish the job). - STATUS: rewritten for the operator, back to one screen.	2026-08-03 19:02:25 +02:00
admin	81d4438630	docs: R-189, R-188 and R-186 closed (agent v0.122.0); R-185 stays open ... - OPEN-ITEMS: all three rows closed with their measurements. R-189 records the migration consequence seen live (a pre-R-189 record has no tier and is not reportable, so upgrading does not retroactively make an old proof visible). R-188 records both directions demonstrated and the fixture cleaned up. R-186 carries the before/after sha values including the CGO_ENABLED discrepancy. The priority list states explicitly that R-185 is the one left open from that group and that this session did not touch it. - capability map: the restore-proof row now says the evidence path itself had a gap — the proof reached the hub only because no restart intervened — and what closed it. - CONTEXT: S-19 (a proof is durable and reportable, a failure is neither, and why) and S-20 (the release ordering, what each step protects, and how a binary is verified). - STATUS: rewritten for the operator, back to one screen.	2026-08-03 16:46:49 +02:00
admin	c102832892	docs: R-86 closed and proven live; ep0 recorded as protected; R-185/186/187 filed ... - OPEN-ITEMS: R-86 CLOSED with the trap in its own wording recorded (the literal reading is never true on a daily tier); R-87 re-ranked UP because R-86 built most of what it waited for; R-185 (the agent cannot list demo-felhom's host backup tier — a missing storage ACL, pre-existing), R-186 (a released binary's sha is not reproducible from its tag), R-187 (R-115's publish leg had never actually run) filed. R-184 was the highest ID in use. - ROADMAP: R-86 collapsed, keeping the reasoning and correcting the shape the row itself proposed — which would have been the never-fires version. - 07-backup-architecture: new contract section — restore-testing is per ARCHIVE GENERATION, with the trap and what did not change (S-1). - 00-capability-map: the unattended restore-proof row upgraded to PROVEN-LIVE on the 635 s due-triggered offsite run, with the restart and teardown evidence. - CONTEXT: S-17 (the rule, the trap, the config key, the hub's derivation) and S-18 (ep0 is Tier 2 — extends D-d's protected list to three machines). Numbered 17/18 because S-14 and S-15 were already duplicated in the file. - STATUS: rewritten for the operator, trimmed back to one screen.	2026-08-03 15:28:32 +02:00
admin	e34b614e5b	docs: R-182 closed, R-90 closed on measurement, R-86 unblocked, ep0 record corrected ... R-182 CLOSED (controller v0.194.0 + hub v0.90.0/.1), proven live on demo-hp. The hub's notification_log for the run reads: two per-app failures RECORDED, one digest SENT naming both, and the customer channel SKIPPED with operator_only. Against the measured previous behaviour — two failures, one email naming one app, one leaving no trace anywhere. Scenario D proved itself on an event I had not planned: disk_critical alarmed on two filesystems, the second was collapsed by the cooldown, and that collapse is now visible WITH ITS KEY. Yesterday it would have left nothing at all. A gap the spec did not anticipate is recorded with its fix: the per-app event also fires from the periodic sweep, outside any run, so making it record-only would have created a NEW silence. The sweep emits a digest too, with no run_id, so it stays under the ordinary hourly cooldown. ep0: MEASURED on the box — 7757 MB (8 GB), 4 vCPU, and the 4 GiB swapfile SURVIVED the resize and is active (checked, because a resize is a stop/start). The 40 GB local disk is UNCHANGED, so no disk figure was touched anywhere. Five documents corrected — three of which the task's list did not name, found by searching. Two audit/evidence documents ANNOTATED, body untouched: they record what was true when written and that is their value. R-90 CLOSED. R-86 unblocked and re-ranked, stated honestly: 8 GB is comfortable, not unbounded — the original OOM was a 14.46 GB restore — so the restore-test cadence should still be paced, just not by fear of the endpoint. target-selection.md's "D-d did not name ep0 either way" is deliberately left standing. It is the operator's question, not CC's. STATUS.md 127 -> 83 lines, items rather than sentences.	2026-08-03 14:00:54 +02:00
admin	b0b269b28d	docs: R-110 + R-115 closed, R-182 re-scoped by measurement, R-183/R-184 filed ... R-182's direction REVERSED by Part 0's measurement. Filed yesterday as "the reserve re-alerts on every status refresh" — too many alerts, seen at the sending end. Measured at the receiving end: 9 events received today, 2 operator emails sent. When two apps are refused in the same second the operator is told about ONE; the other is dropped before LogNotification, so it leaves no row on any channel and cannot be audited. The operator cooldown key is customerID:eventType(+tier) and the capture-failed event carries `app` but no `tier`, so the key has no app identifier. Same failure mode as R-97a, in a second event type that never opted into the narrow fix. Nothing changed — Part 0 was investigation only. Correction owed: yesterday's report said "one recovery_unit_capture_failed per app, HTTP 200". True of what the CONTROLLER pushed; a reader would take it as "the operator was told about each app", which is false. R-110 CLOSED (installer v1.23.0). Both channels moved. The spec's mechanism for channel 2 rested on a factual error — the run-time fetches are sixteen, not nine, and come from felhom-agent, not this repo — so no tag here could cover them; pinned to the agent version being installed instead, on the operator's ruling. Channel 3 needed no change: the URL never carried a ref, so no hub change and no hub bump. R-115 CLOSED. release-agent.sh builds, tags, publishes and verifies by an independent download; check-published-versions.py refuses a tag with no package; CI now runs the full gate set so it actually runs. R-183 NEW+CLOSED: a fresh install fetched the vouched agent binary and its sixteen config files from two different refs, and nothing compared them. R-184 NEW: nothing stops the hub vouching a version that was never released. The R-115 gate cannot see it — measured, the hub manifest and Gitea's package listing are both 401 anonymously. capability map: new PROVEN-LIVE row for the published installer channel. STATUS.md 138 -> 127 lines.	2026-08-03 12:44:08 +02:00
admin	fb652024ea	docs: R-181 closed, R-156 closed, R-110 + R-115 rulings recorded, R-182 filed ... R-181 CLOSED (controller v0.193.0 + v0.193.1) and proven live on demo-hp for BOTH reserve terms. The reserve is now a per-app, per-run ADMISSION decision taken before the app's first write and covering all three write legs, and it gained a size term. The refusal's wording was not weakened; the behaviour moved so it became true, verified by sha256 tree fingerprint. R-156 CLOSED — papra's template mounts the app's own data root. Precondition re-measured rather than inherited (both boxes were wiped today). Part 4, documentation only, nothing built: - R-110 WAITING-ON-OPERATOR -> READY. Ruling: option (b), the installer's publish channel moves to a TAG. Recorded with the condition that decides whether it works at all — it must cover BOTH the /scripts/ git-sync and the nine files the installer fetches from raw/branch/main. - R-115 WAITING-ON-OPERATOR -> READY. Ruling: mechanism (b), a build-side gate refusing to deploy or vouch an unpublished version. The third instance (agent v0.120.0) would have silently downgraded both demo boxes while succeeding. R-182 NEW: the periodic status refresh has no admission scope, so a refused app re-alerts on every poll (measured: a second alert pair 13s after the run's). Pre-existing in v0.192.0; deliberately not fixed in the R-181 task. capability map: the local-backup row moves to PROVEN-LIVE in BOTH halves. ROADMAP: R-165 collapses to CLOSED; R-181 collapsed into it. 07-backup-architecture.md: the reserve's contract stated as what the code provides (S-1 — an architectural contract changed in the same session). STATUS.md trimmed 150 -> 111 lines, "What's broken" no longer holds shipped work, and the stale "After:" line (pointing at work that shipped on 2 August) is fixed.	2026-08-03 11:36:16 +02:00
admin	aa62449694	R-178 CLOSED: both demo boxes reinstalled from the merged golden and proven ... Two boxes, two DIFFERENT supply paths, so the session proved the disk shape and the delivery route rather than one of them twice. demo-hp (layout proof, --golden <local volid>): mp0 at /var/lib/felhom, backup=1, 70G, no mp1; /var/lib/docker and /mnt/sys_drive both real mounts of its subdirectories via fstab; one df figure and one device id (64519) on all three paths; reboots 3/3 with the binds surviving each. demo-felhom (pipeline proof, --force-gitea-golden): fetch_verify succeeding against the vouched manifest for BOTH artifacts -- 'verified sha256 54e2a4c431daf580... matches the hub manifest' for the golden, a7763d31... for the agent. 250G single volume, grep -c '^mp1:' = 0, reboots 3/3. Journey proven on both, endpoint-level: claim -> deploy -> back up -> restore, with a planted marker returning byte-identical on each box. Ceiling measured gone: 65 GiB and 233 GiB available to a recovery unit, against 19 and 45. R-165 -> IMPLEMENTED, not PROVEN-LIVE, on the operator's ruling. B2, which that row records as the bulkhead's replacement, fired live for the first time and does refuse per app, delete nothing and alert -- but it is checked only in captureAllRecoveryUnits while runVolumeDumps writes the bulk unguarded, and its 'the previous unit is untouched' claim was measured false (182,272 B dump replaced by 2,147,666,432 B under a manifest still dated 06:34:26). -> R-181. New: R-179 (uninstall leaves NAS network-storage units), R-180 (--archive-storage not cross-checked against the ACL grant; 403 at step 8/8 after root@pam is rotated), R-181. Third instance of R-115 recorded (agent 0.120.0 unpublished). No code written, no version bumps -- this was a runbook.	2026-08-03 09:34:15 +02:00
admin	14d8c00781	docs: R-165 merge built and proven at the bake; R-163 + R-175 closed, R-178 filed ... 07-backup-architecture.md gains §7.5.1 (S-1: the contract changed in the same session): the ceiling §7.5 describes no longer exists for a box built from golden >= 0.192.0, the bulkhead's replacement is recorded, and R-175 is FIXED here rather than left standing — the bound is restated as a function of mp1 and scoped to split-layout boxes, naming all three real shapes. Capability map: new row as IMPLEMENTED, deliberately NOT proven-live, with the missing leg named — no box has been reinstalled from the golden, and "the golden baked" is not "a box built from it works". R-163 CLOSED: the ceiling it recorded stops existing. R-176(a) answered by P1; (b) WITHDRAWN, since every node is reinstalled rather than migrated. R-178 filed for the reinstalls, which were not done this session. CONTEXT S-13 (the variant chosen on measurement; pruning rejected with its reason) and S-14 (prove first, then vouch — the golden is published but deliberately unvouched, because vouching is what makes a fresh install pick up a layout no box has been proven from). STATUS: plain-language section; both operator questions now answered, so the waiting-on-you item is cleared. Two older entries trimmed so the page did not grow.	2026-08-03 07:16:15 +02:00
admin	41dbecb264	docs: R-167 + R-158 CLOSED, R-165 SPIKED, R-174..R-177 filed ... R-167/R-158 shipped and proven live (controller v0.191.x, hub v0.89.0): two new capability-map rows PROVEN-LIVE with live citations, and 07-backup-architecture.md §7.5's closing claim "nothing warns when an app crosses the line" is now false and rewritten (S-1: an architectural contract changed in the same session). §7.5 also gains the caveat that its size bound is ONE BOX'S, not the fleet's. Part 3 SPIKE (audits/SPIKE-r165-mp1-merge-2026-08-02.md): M1-M5 measured, NO layout touched. Three findings the merge session must not re-derive: "the layout" is not one thing (200G/50G vs 50G/20G vs 16G/8G); mp1 is a BULKHEAD and not only a ceiling, so after the merge an overflow reaches /var/lib/docker; the golden fails closed on the split in four places. D-a's condition (1) is currently SATISFIED — no external box is in the hub's register, and both demo boxes are Tier 0 and reinstallable. Recommendation given, choice NOT made — it ends at the operator's ruling. CONTEXT.md S-11 (D-c's routing, and why R-158's own backup_failed proposal was overruled) and S-12 (the monitoring landed BEFORE the merge). STATUS.md gains the plain-language section and the merge decision, with two older entries trimmed so the page did not grow. New rows R-174 (closed same session), R-175, R-176, R-177; each ID grepped free before minting.	2026-08-02 23:56:16 +02:00
admin	ad28699761	docs: R-157 A / R-170 / R-171 closed — boot recovery finished ... Controller v0.190.0. Docs only here; no hub change, no hub version bump. - audits/DIAG-bootrecon-drive-absent-2026-08-02.md — NEW. The Part 0 diagnosis, including the run that produced a FALSE NEGATIVE and the mechanism behind it (the agent re-binds an unmounted drive within ~60s, so the drive gate's startup reconcile restarted the apps one second before the sweep looked). Records that the write hazard was blocked only by an ACCIDENTAL filesystem permission that no code owns and no test pins. - architecture/02 §0a — the boot-recovery contract (S-1): both gates read desired state; the sweep observes a SETTLED fleet and each sample must refresh first; nothing is started without asking, fail-safe. Plus the durable warning: Manager.StartStack has no gate of its own. - 00-capability-map — the boot-recovery row, with the repeat count cited per N.5 (6 of 6 hard resets) rather than a bare PROVEN-LIVE. - OPEN-ITEMS / ROADMAP — R-157 CLOSED (both mechanisms), R-170 CLOSED, R-171 NEW and closed the same session, marked a regression from v0.189.0. - STATUS.md — the power-cut line moved from "What's broken" to "What works right now" with its repeat count; one dated bullet in the change log. - CONTEXT.md S-13 — the lessons worth carrying: "it didn't happen this time" is not a disproof; widening a window makes previously-unreachable overlaps reachable; and a settle detector is only as good as the freshness of what it samples — the fix's own defect, found live rather than by review.	2026-08-02 20:38:21 +02:00
admin	5c97fbc397	docs: R-166 SHIPPED — the desired/in-flight/observed split (D-b) ... Controller v0.189.0 implements operator decision D-b. Docs only here; no hub change and no hub version bump. - architecture/02-controller-module-map.md §0a — NEW, and it is the S-1 contract: desired (app.yaml) / in-flight (own marker file) / observed (not persisted), with the rule that ties them — never derive one from another. Absent desired state means UNKNOWN, never "running". One file, one writer. D-b's binding safety rule quoted verbatim. - 00-capability-map.md — the boot-recovery row now rests on a recorded signal, with the three live flows from 9201. The interrupted-operation half is marked IMPLEMENTED, not PROVEN-LIVE: nobody killed the controller mid-backup on metal. - OPEN-ITEMS/ROADMAP — R-166 SHIPPED with both blocking facts and their answers; R-157 mechanism B CLOSED and A restated as the whole item; R-170 NEW (the drive-backed boot gate still infers a Stop from a container count). - STATUS.md — the "an app can stay switched off and nothing says so" line rewritten to what is actually left: timing. - CLAUDE.md — end-of-session checklist gains: confirm your own last push's CI run went green, BY RUN ID. The failure email is a push signal; this is the pull check. - CONTEXT.md S-12 — the rulings, and the two lessons worth carrying: a test that constructs the thing it should prove the caller constructs is hollow (its red-proof will say so), and a field-by-field struct rebuild in a save path is a defect on sight.	2026-08-02 18:58:27 +02:00
admin	c718aad1bc	docs: R-168 SHIPPED, R-29 CLOSED on the demonstrated alarm, R-169 minted ... SPIKE-ci-runner-2026-08-02.md: all six probes with method, measurement and ruling; none STOPped. P2 (stock image has git but no python3) and P6 (a runner that loses its state re-registers and orphans the old record) changed the design; P5 (a failed run signals NOTHING) is why the alarm exists at all. R-168 SHIPPED with its evidence. R-29 CLOSED — on the demonstrated alarm and not on a green run, as required: the class it opened is answered at both ends, the hook refusing locally and CI catching a --no-verify bypass and emailing. R-161 noted: its automatic half now exists for the STATIC gate, while its original scope, the runtime gate, is deliberately still not automatic and should stay that way. NEW R-169 (grep established R-168 was the highest in use): CI can only report, because there is no gate in the road. Making it blocking needs branch protection plus a PR workflow, both of which change how the operator works — so it is theirs to decide, and the row states the cost honestly rather than recommending it. CONTEXT gains S-8 (CI detects, does not block, and why that is structural), S-9 (a detector that tells no one is not finished, plus the curl and Cloudflare-1010 traps), S-10 (the runner is unprivileged because DooPlex is Tier 2), S-11 (CI reproduces the sibling layout). CLAUDE.md gains the rule earned by red-proofing: a go test -run pattern that matches no test prints ok and exits 0, and an instrument that can silently drop results is not a measurement.	2026-08-02 16:35:34 +02:00
admin	4707be755c	docs: R-94 closed, R-29 leg (a) closed + leg (b) half, R-168 minted ... hub/CHANGELOG v0.87.0 + scripts/CHANGELOG gate-enforcement entry. CONTEXT gains S-6 (the hub renders no host-install version and the gate pins its absence) and S-7 (gates run from one entry point per repo; reuse_refs_check was fixed rather than the REUSE.md convention, with both rejected alternatives recorded). OPEN-ITEMS: R-94 CLOSED all three legs, leg (a) by DELETION with its reason; R-29 leg (a) CLOSED and leg (b) HALF-SHIPPED with the census result written into the row (13 gates; every gate a CLAUDE.md names was green, two of the four unnamed were red); R-161 gains its successor pointer. NEW R-168 (grep established R-167 was the highest in use): Gitea Actions runner — measured 2026-08-02 as Gitea 1.26.2, Actions enabled on all four repos, 0 runners, 0 workflow runs, 0 branch protections, and the consequence that trunk-based direct-to-main pushes leave no merge for a status check to gate, so CI here can detect but not block. BLOCKED on a spike over host-mode vs privileged DinD on DooPlex and whether the workflow can avoid JavaScript actions. ROADMAP: R-94 collapsed to its one-liner, R-29 updated, R-168 added.	2026-08-02 15:28:31 +02:00
admin	e994bf35d2	STATUS.md: a plain-language operator page, and today's four decisions recorded ... Documentation only — no code, no box, no build. STATUS.md (repo root, 652 words / 67 lines): what works · what's broken · what we're working on · waiting on you · changed since. A VIEW of OPEN-ITEMS.md, holding nothing of its own; not CONTEXT.md, and both files now say why they stay separate. No R-n is the subject of a sentence — identifiers are bracketed pointers only. CONTEXT.md S-5 records the four operator decisions taken 2026-08-02 (D-a … D-d), none of them implemented: D-a merge mp1 into mp0 rather than resize it — before any external install, and D-c ships in the same step → R-165 D-b desired/observed app state in its own store, with the state-store safety rule verbatim → R-166 (BLOCKED) D-c customer fill warning + operator backup-failure alert → R-167 D-d only DooPlex and Peti's box are protected → target-selection.md R-163 RE-FRAMED, not closed: the sizing question is withdrawn rather than answered; the row survives as the record of the constraint until R-165 lands. R-156's papra referral RESOLVED — deployed nowhere, so the template fix strands nothing; the docker ps evidence is recorded with its provenance and its scope limit. target-selection.md: two protected machines, everything else disposable. ep0 is no longer Tier 2 but is not scratch (it holds the only off-premises copy of real customer data) — flagged for explicit operator confirmation. The demo-box backup-target fence drops from prohibition to stated cost, because D-d spends that reference anyway. CLAUDE.md gains an End-of-session checklist carrying the STATUS.md maintenance rule and "a finding goes in OPEN-ITEMS.md first".	2026-08-02 14:20:29 +02:00
admin	1956e5d390	hub v0.84.0 — break-glass console credential on the host page ... The credential existed and was not reachable when it was wanted. Every box has had a strong random root@pam password since TASK G1, vaulted in the hub at day 0 and used for real during the sshd incident — but the only way to read it back was a hand-written curl carrying the global operator key, a secret kept out-of-band. In practice the PVE web console on a demo box felt locked. The host page grows a Console access card: presence + username + set_at by default, Reveal fetches the plaintext on demand for 60 s with a Copy button. Masking clears the JS variable, and also fires on a second click and on visibilitychange. A host with nothing vaulted says so, and says why. The secret is NEVER rendered into the page, and that constraint shapes the change. The render path uses a new store.GetHostRecoveryMeta whose struct and SELECT both omit the secret column, so it is structurally incapable of carrying one. The plaintext crosses the wire only in the response to POST /hosts/{id}/reveal-recovery-credential (Cache-Control: no-store, CSRF-gated at the ServeHTTP level; POST precisely so that gate applies and so no secret is retrievable by URL alone). Deliberately NOT the customer page's data-secret widget, which embeds the plaintext on every load. A delivered reveal writes one recovery_credential_revealed event on the host's customer timeline (info, source hub, Hungarian) via SaveEvent alone — no dispatcher, nobody emailed, the log_tail_requested shape. Two reveals write two events: the register records accesses, not states. A 404 is not an access. An unbound host reveals fine and writes no event; the [INFO] hub line, carrying the username and a length only, is then the record. The global-key API path is untouched by design — it is the route for when the hub UI itself is broken, and coupling it to the session layer would delete the independence that makes it a fallback. Recorded as a real trade: the hub session password alone now unlocks console root fleet-wide, where retrieval previously also needed the global key. Accepted for a single-operator, HU-geo-fenced hub that already stores these passwords in plaintext at rest (CONTEXT.md ruling S-4). The plaintext-at-rest half is filed as R-133 — every hub DB backup is a fleet-wide console-credential dump. Tests 550 -> 559; four red-proofs (page leak, audit event, CSRF gate, route order) each run, observed failing, and reverted. The route-order proof is a seam test driving ServeHTTP: a handler-level test cannot see that defect, because the handler is correct and simply never runs.	2026-07-31 08:19:36 +02:00
admin	de5a3e5765	docs: retire the last two false gate-enforcement claims; scope the ranking heading ... Closes the record-hygiene rider. Part 3 of the spec (documenting a ROADMAP/OPEN-ITEMS state convention) is deliberately NOT done — its stated evidence is false; see REPORT-record-correction-2026-07-29.md. - CONTEXT.md:540 — "scripts/hub_confirm_gate.py enforces" was present tense about a gate invoked by nothing. Now says it asserts but is not enforced (R-29). Third instance of the class after :564 and configs.go:27. - REUSE.md:62 — same claim, "enforces zero". The RULE stays (never native confirm()/prompt() is correct guidance and this is a reuse-reference row); only the enforcement claim changes, and it now says the rule holds only as long as you keep it. - OPEN-ITEMS.md:4 — root REPORT.md is the overwritten per-session file; REPORT-<topic>.md is the non-clobbering sibling form (CLAUDE.md:82-87), of which 14 exist. The prohibition on durable content living only there stays. - OPEN-ITEMS.md:55 — "Why the READY rows rank this way" promised a complete ordering and listed 5 of ~15 open rows. Scoped to TOP, with a half-sentence saying it is deliberately not a full ordering. No row added to the list. hub/internal/web/configs.go:27 left alone (R-94 leg (b), needs a hub build). No gate wired, run or fixed. Documentation only, no version bump, no CHANGELOG.	2026-07-29 11:22:07 +02:00
admin	7383400a23	docs: file R-29 to the register; attach the gate-orphan instance to its class ... d4c07873 filed "hostinstall_gates.py is invoked by nothing" as a novel observation. It is not novel — R-29 already names the class (green gates are enforced nowhere; one sat RED for 16 releases while every REPORT said green), and R-29 was missing from OPEN-ITEMS.md entirely, having never been carried across the 2026-07-27 register rebuild. An open item about work not getting done was absent from the page that decides what gets done. Ruling on whether R-29 is the right home for a non-design-v2 gate: YES. Its title says design-v2, but its own audit list already spans mount-safety, secrets and dedup gates across four repos, and its part (b) — "the systemic half is the real item" — is about the enforcement mechanism, which is gate-agnostic. hub_confirm_gate.py is already on its list and sits in the same scripts/ directory. No new ID minted; R-29's own text forbids it, and this is the third re-raise it has absorbed. - OPEN-ITEMS.md: open R-29 (READY, S(a)/M(b)), with the orphan evidence and the two separable parts R-29 already defines. - OPEN-ITEMS.md: R-94 leg (b) now points at R-29 as its class. - ROADMAP.md:158: audit list extended with hostinstall_gates.py (RED today, 1.19.0 != 1.22.0) + hub_confirm_gate.py verified orphan. Entry not rewritten. - ROADMAP.md:147: cited a non-existent R-164 — it means controller v0.164.0. - CONTEXT.md:564: asserted in the present tense that the version cross-check is "gated by scripts/hostinstall_gates.py". It exists, is red, and runs nowhere. - OPEN-ITEMS.md: READY #1/#3/#4 markers dropped — they duplicated ranked-list positions and the gap was left by the row merged in d4c07873. - OPEN-ITEMS.md: E-2d citation :322-341 widened to :322-343; the invocation it describes is at :343, two lines outside the old range. - backlog/README.md: two-line lead naming OPEN-ITEMS.md and ROADMAP.md. - REPORT-record-correction-2026-07-29.md: the report CLAUDE.md:82-87 requires for both commits. Root REPORT.md (E-2 increment 1) untouched. No gate wired, fixed, run or deleted — that is R-29 part (b), its own task. Documentation only. No version bump, no CHANGELOG entry, no box touched.	2026-07-29 11:13:23 +02:00
admin	070b0ce2dc	07-backup-architecture: rewrite as the recovery model (D1-D6); D5 BLOCKED by R-108 ... Replaces the 2026-07-14 DRAFT entirely - it was verified against controller v0.132.0, 51 versions stale, while being cited as authoritative. Now verified against controller 0.183.0 / agent 0.110.0 / hub 0.80.0 / catalog 4252121. NOT ratified. Every statement is labelled [DESIGN] (a decision from the architecture discussion) or [FACT] (file:line, live output, or a citation to _recovery-inventory-2026-07-28.md). Phase 0 verified D5's premise and it FAILED. Ten read surfaces were checked; nine are clean (SMB under both namespace shapes, FileBrowser for local drives, .fab import and download, /api/debug/*, all three ServeFile sites, registering the backup dir as a drive). One is not: FileBrowser binds a NETWORK share at its ROOT while local drives are userdata-scoped, and no IsNetwork() filter guards the deploy dropdown, the per-app migrate targets, or handleStorageMigrateApp. So an app namespace - and under D5 its plaintext secrets - can land somewhere a browsing surface serves with download:true. Not a leak today (the unit's app.yaml is secret-stripped); it becomes one under D5. D5 is therefore written as a BLOCKED target, not as settled. -> R-108 Core artifact is the failure -> recovery matrix (15 rows). RTO cells carry only measured durations; 12 are blank and each blank is listed explicitly as a finding. RPO cells carry the configured cadence that bounds RPO, labelled as such - no RPO has ever been measured from an incident and none is invented. The 7/53 vs 9/43/1 Tier-2 coverage counts are both recorded, unresolved; neither adopted. Coupling (S-1): capability map section C now cites the matrix instead of restating routes; ROADMAP gains R-102..R-108; OPEN-ITEMS gains only the in-flight R-108 plus C9-F1b/F3/F4 -> R-n cross-references; CONTEXT records D1-D6 as standing ruling S-3. No code, no version bump, no deploy, no box touched.	2026-07-28 20:50:41 +02:00