379 Commits

Author	SHA1	Message	Date
admin	e5eee501b5	R-331 (controller half): forward stats_known so the hub can tell empty from unmeasured (v0.225.0) ... The hub's operator Backup card read `Snapshots 0 / Repo Size 0 MB / Integrity Unknown` for EVERY customer, because it rendered the report's `backup` object -- whose snapshot/size/integrity fields have had NO producer since disk-tier restic moved to the host agent (slice 8C). buildBackupReport leaves them zero deliberately and says so. Measured on demo-hp 2026-08-30 while that night's log said `[offbox] backup OK: 8 app(s) backed up, 67 snapshot(s), 2m14s`. The live numbers were always in the report's `offsite` object, which the hub already reads for its Offsite page and its fill/staleness alarms. The hub fix is to render that -- and that made exactly ONE field mandatory that was not being forwarded. snapshot_count:0 means two opposite things: "holds nothing" and "never measured". R-225 measured that confusion inside this repo (a rebuilt box rendered 0 pillanatkep over a store really holding snapshot f3d9cd67), and settings.OffboxTarget.StatsKnown fixed it for the controller's own UI. It was never put on the wire, so the hub was free to make the identical mistake one layer up -- and did. OffboxReportStatus.StatsKnown now carries it, omitempty, so an older controller sends no key and a reader degrades to UNKNOWN, never to EMPTY. Absence is ignorance, not emptiness. The four dead BackupReport fields stay on the wire (historical reports in the hub store must keep parsing) but now carry a warning naming R-331 and pointing at Offsite. TestBackupReport_DeadFieldsStayZero fails the moment a producer appears for one -- the prompt to update the hub card in the SAME change rather than ship a field nothing renders. RED-PROOF: drop `StatsKnown: t.StatsKnown` -> "a MEASURED empty repository reported stats_known=<nil>". Tests assert the JSON the hub sees, not the Go struct: measured-empty and never-measured must differ ON THE WIRE, which is the entire point of the field. Green gate clean: 28 packages, rc 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LB8FmJaGd2cyjvy6dbEjpM	2026-08-30 18:38:10 +02:00
admin	45b52b6ed5	R-330: live validation on demo-hp — 3 scans inside the window, 0 events ... v0.224.0 deployed to both demo boxes (both `0.224.0 … (healthy)`). Proven through POST /api/backup/run, the endpoint the UI button invokes: 8 stacks stopped and restarted over 87s, three dead-app scans ran INSIDE that window (16:09:26 docmost, 16:09:56 paperless-ngx, 16:10:26 romm -- the same three apps that alarmed the night before on 0.223.0), zero app_start_failed pushed. The scan count is the positive control, not decoration: an absent alarm is equally consistent with "suppressed correctly" and "the scanner stopped". A first run is discarded IN THE REPORT rather than quietly dropped -- it fired 52s after a controller restart, inside deadAppBootGrace (90s), where the scan returns early and could not have alarmed whatever the code did. demo-felhom is deployed but NOT independently proven and says so: its single app cycles in ~1s, too fast for any 30s scan to land inside. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LB8FmJaGd2cyjvy6dbEjpM	2026-08-30 18:12:26 +02:00
admin	92cebb8c95	R-330: stop the backup alarming about the apps it is holding down (v0.224.0) ... Measured live on demo-hp 2026-08-30 (controller 0.223.0): the nightly db-dump and offbox-backup legs stop each stack ~13s to tar its volumes while the deadapp-check job scans every 30s, so the scan caught whichever stack was mid-cycle and pushed app_start_failed to the customer. 61 e-mails about apps that were never broken. The defect is not a missing mechanism. quiesce/suppress.go solved exactly this in v0.179.0 and works -- but classifyRunStates read only the quiesce loop's set, and that loop covers the WHOLE-GUEST backup. The per-app legs stop stacks through Manager.DumpAppVolumesSafe, which registered with nothing. Two mechanisms stop apps on purpose; only one told the alarm. Fifth instance of the "seam built but never wired" class, and the first where the unwired half was a consumer. The suppression now rides AppStopGuard, which already brackets every deliberate stop in the product (Begin before the stop, End after a successful restart) at all three call sites, and which main.go hands as ONE object to the backup manager and the exporter. scanDeployedAppRunStates takes the union of both sets. All three per-app stop paths are covered, not only the reported nightly one. It cannot latch -- End() runs only on a restart that SUCCEEDED, so unlike the quiesce loop an open-ended hold is a real hazard here: 1. ReleaseFailed drops the entry IMMEDIATELY on a restart that broke, wired at every failure path, so the app alarms on the next scan; 2. Begin REPLACES the set (one marker file = one operation); 3. appStopMaxHold (6h) caps a hold nothing released, logged at WARN. Grace is 180s, deliberately quiesce's own constant and derivation. Suppression is NOT persisted: after a crash the guard holds nothing and a down app must alarm. ReleaseFailed keeps the durable crash marker; a test pins that. Three companion red-proofs, each printing the pre-fix value (REPORT.md section 5): - drop markStopped from Begin -> "suppressed at stop = map[]" - drop ReleaseFailed from the dump -> "map[bookstack:true] after a restart that FAILED" - pass nil instead of appStopGuard -> the AST wiring test fails The third is load-bearing: the component was never the broken part, so a suite that only injected it would have been green against the shipped defect. Green gate clean: go build + go vet + go test ./... -- 28 packages, rc 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LB8FmJaGd2cyjvy6dbEjpM	2026-08-30 17:58:15 +02:00
admin	9832760027	v0.223.0: the app-down alarm reached nobody (R-329), and the stop nobody heard (R-386) ... R-329. NotifyAppStartFailures emitted severity "warn". The hub accepts exactly {info, warning, error, critical} and silently coerces anything else to "info", which severityNotifies then drops BEFORE both legs. Banner shown, event stored, POST 200, no mail sent. One word. This is the second time: DiskAlertKind.Severity emitted "warn" until v0.215.0 and its own comment records that every warning-level disk alert went to nobody. A comment recorded the lesson and nothing enforced it. The guard is now an AST walk over the whole controller - grep cannot work here, since "warn" appears legitimately nine times as a healthcheck status vocabulary. The sweep found exactly one bad severity. Its limits are stated: the walk cannot follow a variable, so all six dynamic call sites are registered by name with the values each can take, and a new one fails the test. Two of the six were found by the guard, not by the hand sweep before it. Also pinned: fillwatch.Band.Severity() returns "" for BandOK, which would vanish the same way. It is unreachable because Check() notifies only on escalation - but that safety lives in a different function from the one that looks unsafe, so the test asserts the consequence rather than the mapping. app_start_failed gains a customer toggle, DEFAULT OFF, per operator ruling. The operator is mailed either way: processOperator never consults customer prefs. It is deliberately NOT in operatorOnlyEvents, which would make the toggle a lie. R-386. classifyRunStates decided "the customer stopped this" from the STATE, so every stopped stack was assumed deliberate. Measured on demo-hp: privatebin stopped out of band, nine scans, zero events, zero banner - while the comment beside it claimed an out-of-band stop still alerts. DesiredState already records the answer and has exactly one writer. Stopped -> no alarm; Running -> alarm; absent -> UNKNOWN, keep today's behaviour AND say so. Absent stays silent deliberately: reading it as "nobody asked" would email about every app anyone ever stopped, fleet-wide, on the first cycle after upgrade. The gap is bounded not silent - IntentUnknown is set and the names are logged at INFO on the heartbeat cadence. failedRestart still lifts a Stopped intent, or F-CRIT-1 re-opens. No new DesiredState writer. Two settings toggles each governed two alarms. "Lemez figyelmeztetes (90%+)" also wrote disk_critical, the drive-is-FAILING alarm. Now four honest toggles; 12 became 15. A no-op save stores the existing slice verbatim, so byte identity is by construction - without that guard the defaults case reorders, which the red-proof caught. Test count 1504 -> 1522. Five red-proofs, five seen failing; one passed first time and is reported - that mutation was inert, not the test weak.	2026-08-23 11:21:06 +02:00
admin	5da11c4480	v0.222.0: ask whether a supervised member is DEAD before whether one is UNHEALTHY (R-384), and stop promising an undo copy nobody looked for (R-383) ... R-384. aggregateState returned StateUnhealthy the moment unhealthy > 0, and the R-51 mixed-case block that asks "is a supervised member dead?" sat below it. A two-container app whose database exits goes unhealthy BECAUSE it cannot reach that database - so the symptom the dead database causes was what suppressed the alarm for it. unhealthy is not a down state, so classifyRunStates never marked the app down and app_start_failed never fired. Measured live on demo-hp 2026-08-22: bookstack-db stopped at 21:27:01 and the F-OBS heartbeat printed "0 currently down" throughout. R-51's 18-hour immich failure, back through a different door. Two things moved, and either alone leaves the defect standing: the supervised test is hoisted above the unhealthy/starting/restarting returns, and "some members are up" now counts ANY member not in the down bucket. The old guard was running > 0, which made the R-51 block unreachable in exactly the case it was written for. IsDownState is byte-identical - unhealthy stays excluded, because an unhealthy container is running and folding it in reintroduces the flapping that exclusion exists to stop. No new state was minted. Only the ORDER changed. The priority comment was rewritten because it asserted an ordering the code no longer has. Three subtests in TestAggregateState_UnchangedBranches were AMENDED: they asserted an unhealthy/starting/restarting member beat an exited peer on unless-stopped, which pinned the defect as settled behaviour. They keep their intent with the down member given a benign policy. R-383. The double-failure message said the previous state's backup EXISTS, built from the returned path without asking the filesystem - and a missing file is one of the two ways that rollback fails. undoCopyPhrase now describes the copy from disk: present, partial, missing (still naming where it should be), or never written. Zero-length counts as missing. Test count 1494 -> 1504. Four red-proofs planted, four seen failing; the two halves of R-384 convict independently.	2026-08-23 07:25:00 +02:00
admin	da75603553	R-385 record: v0.221.1 gets its own CHANGELOG heading ... 0.221.1 was built, baked and vouched on 2026-08-23 with no entry of its own. The prune-ordering fix (commit 810b18a) was written inside the v0.221.0 entry, so the newest heading in the record named 0.221.0 while the fleet ran 0.221.1. The image was never in question - the record was. The reasoning is MOVED, not rewritten: the paragraph is verbatim from the v0.221.0 entry, which no longer claims it. The new entry also names the pinning test and its red-proof. Pushed alone and before anything else this session, so the record is correct even if the rest of the session does not land.	2026-08-23 07:06:18 +02:00
admin	810b18ab8e	R-361 follow-on: the undo-copy prune must run above the already-current check ... Excluding pre-restore-* from db_dumps made that list stable across restores, so CaptureRecoveryUnit's already-current early return began firing where it never had - and the prune, which sat after it, stopped running in exactly the case it exists for. Measured on demo-hp minutes after the change: four undo copies on disk against a cap of three. The prune is housekeeping on the dump directory and is independent of whether the manifest needs rewriting, so it belongs above the check. Pruning cannot disturb dbDumps, which no longer contains those names. Pinned by TestR361_UndoCapHoldsWhenTheUnitIsAlreadyCurrent; its red-proof moves the call back below the return and the cap fails at 5.	2026-08-23 00:02:32 +02:00
admin	968c968559	R-361: the safety dump destroyed the app's own database backup ... writeSafetyDump called DumpOne into the app's OWN unit dir and renamed the result to pre-restore-* afterwards. DumpOne writes <stack>-<dbtype>.sql - the app's canonical dump - so every safety dump overwrote the app's real backup and then moved it away, leaving the app with no database backup until the next nightly run. A local restore-from-unit in that window tells the customer the app never had a database. The comment beside it asserted the rename meant it 'can never overwrite the app's real dump'. False as written, and believed for four months. Measured live before the fix: docmost and bookstack each held only pre-restore-* files and no canonical dump. DumpOneTo takes the final path and derives its own .tmp from it. DumpOne keeps its signature and calls it with the canonical name. writeSafetyDump asks for its own name directly; the rename is gone; the comment now states the invariant and how it is enforced. db_dumps no longer lists the undo copies. All three consumers of Manifest.DBDumps were grepped and named - all inside recovery_unit.go, none reads it for recovery. The files are neither deleted nor hidden. Tests 1485 -> 1493. FIVE red-proofs, TWO PASSED first time and both are reported: the behavioural tests inject the dump seam so a mutation inside DumpOneTo was invisible, and 1.3 had no test at all. Guards added at the layer each defect lives in; both mutations then convicted.	2026-08-22 23:39:59 +02:00
admin	1a2405e86f	R-379: --clear-restore-hold now states the required restart ... It runs as a second process: it clears settings.json but the running controller keeps its in-memory copy and goes on refusing. Measured on demo-hp - clear succeeded, file correct, start button still refused until a restart. Also records the lost-update window between the two processes, and why clearing through the running controller (the right shape) needs an operator tier the controller's HTTP surface does not have.	2026-08-22 18:21:47 +02:00
admin	5b52a5964d	R-379 fix: the rollback must re-discover the DB container ... Found by v0.220.0's own live walk on its first real run. writeSafetyDump captures its DiscoveredDB before the stop; the DB-only start then re-creates the container with a new id, so the rollback's docker exec hit a dead container and sat in waitDBReady for 30s. The app was held for an infrastructure reason while its data was recoverable. Re-discover and match by {stack, engine} - what reimportDBDumpsFrom already did. Fail closed when the container cannot be found. No unit test caught it because they all inject the import seam and never look at container identity. The new test asserts the identity handed to the import.	2026-08-22 18:14:31 +02:00
admin	2c724c9283	R-379/R-380: put the customer's undo copy back when a database restore fails ... R-379 and R-380 were one failure. Both ended with a half-restored database; the only difference was whether it looked broken. Postgres emptied and crash-looped; MariaDB applied part of the dump and reported health=healthy with a zero-row schema-version table. Measured live on demo-hp 2026-08-22. The undo copy was already taken and already good - proven by hand that day on both engines. Nothing in the product could apply it. Now it does, with the same ImportDump call, before any restart and inside the DB-only window. The WHOLE undo set, matched on this run's stamp. writeSafetyDump returned one path for an app with two databases; a rollback on that would restore one and leave the other half-written. When the rollback also fails the app is HELD STOPPED (operator ruling): a running app on a half-written database lets the customer make the damage permanent. Every start path refuses it - customer button, appstop Recover, boot sweep - via the shared driveStartGate, checked ABOVE its driveless early return because these apps have no drive. The marker is ended so nothing auto-restarts it. The row goes red. Cleared with --clear-restore-hold, an operator CLI route. --single-transaction is a belt on Postgres only; MariaDB DDL is not transactional and that is why the rollback is the fix. R-381: the engine's stderr stops reaching the customer (615 bytes on MariaDB, its middle rows out of their own database) and starts reaching the operator log, which never had it. R-382: the summary log prints the volume count it already held. Undo copies resolve to their own app, are marked IsUndo, and are capped at 3 per app, pruned from the capture side. The reported render-as-an-app symptom did NOT reproduce - the live page was read first and had zero occurrences. Tests 1468 -> 1483. Eight red-proofs; ONE PASSED and is reported: the R-381 behavioural test injected below ImportDump. A guard at that layer now convicts.	2026-08-22 18:03:18 +02:00
admin	08eb1a6e3a	R-356: the off-site restore refused every app that has no data drive ... ReconstituteFromOffsite and PlaceOffsiteRestore both resolved the restore destination with the RAW HDD_PATH and read an empty answer as "the app is not installed". For 40 of the 53 catalogue apps that answer is correctly empty and permanent, so both actions refused forever for a running, healthy app — and told the customer to reinstall it "in the same place", which those apps never offer. Separate the two questions. "Installed?" is asked of ListDeployedStacks via a new Manager.isStackDeployed that fails CLOSED on a nil provider. "Where?" is answered by GetAppDrivePath — the same resolver CaptureRecoveryUnit wrote the snapshot with, so the restore aims at the place the backup came from. The 13 drive apps are unchanged: own drive, mismatch check, ack still required. A third refusal, with its own sentence, covers installed-but-no-resolvable-root. Fixtures that marked an app "installed" by giving it an HDD path now state deployment as its own fact. No assertion weakened.	2026-08-22 13:08:46 +02:00
admin	5ce3a44645	v0.218.0: attribute a DB container by its compose project, and replay volumes on the off-site restore ... R-355 (first, because it is the only one where data can be lost for good). paperless-ngx's PostgreSQL was dumped into backups/primary/paperless/db-dumps/ — a directory for a stack that does not exist, on the system drive — while the app's own unit recorded db_dumps: null. The same misattribution reached writeSafetyDump, so a destructive restore of that app took NO undo copy and the fail-closed refusal was never reached. Fixed by reading the compose project label, which is the stack name by construction (compose runs with cmd.Dir set to the stack dir and no -p). The old derivation stays as the fallback and an unresolvable attribution is now loud. Catalogue sweep, proven able to convict: one affected app of 53. The fix is in the controller, not the catalogue. R-354. ReconstituteFromOffsite skipped every unit placement and the volume archives live inside the unit, so the off-site restore had no volume leg at all — proven live with planted files: calibre-web's 1,422,848-byte config archive was in the unit, the snapshot and the checking folder, and the restore reported success without it. For the 40 of 53 apps that declare no data drive that archive is the whole dataset. restoreDockerVolumesFrom is the local path's own replay with an explicit directory: ONE implementation, two callers. Volumes replay before the database and inside the stopped window. VolumesReplayed reaches the message. The comment beside the skip was half false and is corrected; the half that still holds — the live unit is the local path's source — is named, and scenario D fingerprints the whole live unit across the operation. Seven red-proofs, each asserted applied and reverted. Two found defects in the tests, not the code: scenario D passed with the unit guard removed because the fingerprint had been narrowed and was blind to the unit root.	2026-08-22 09:43:22 +02:00
admin	f94543ee5c	v0.217.0: prefill from the app's own backup, where-the-data-goes on deploy, bounded inventory fan-out ... Completes R-351 and ships R-352's visibility half. Gates 11/11 OK, suite 28 packages ok, go vet clean, -race clean on the changed package - all run and read BEFORE this commit. PART 2 SCENARIO A - the deploy page prefills the address and data folder from the app's OWN backup. backup.RecordedUnitForStack scans every readable namespace root (the app is NOT installed in this case, so there is no own drive to ask) and reads manifest.json plus the captured compose/app.yaml. Local file reads only: no network, no restic, no restore. RecordedAddress.Known() requires BOTH halves on purpose - an absent SUBDOMAIN makes the live deploy path substitute the CATALOG default (stacks/deploy.go:88-90), and offering that back as "what your backup says" would be a fabricated fact. The prefill is labelled as coming from the backup and stays editable: a memory, not a lock. PART 1 VISIBILITY (R-352) - the deploy page now states where the app's data will live before the button is pressed. Measured 2026-08-21: 13 of 53 catalogue templates declare a storage field; the other 40 have none and their data goes to the system drive, which no screen said. Metadata.HasDeployField answers "does this app have somewhere to PUT a recorded value?" - for the 40-class a recorded placement is a fact to state, never a value written into a field that does not exist. NO PLACEMENT CHANGED. NOTHING MIGRATED. The rest is a filed specification. PART 4 - measured before theorising, on the live off-site target: snapshots --json 2605 ms once; stats 2697 ms PER APP, sequential, 5 app tags => 2605 + 5*2697 = ~16.1 s, matching the reported ten-to-fifteen seconds. The cause is the shape already on file, so the per-app size calls now run concurrently, BOUNDED TO 4. The bound is the safety property, not the speed one: the repository is a Hetzner Storage Box with a session cap, and a refused size call returns SizeBytes 0 - a silent UNDER-REPORT of the customer's data rather than a visible failure. Peak-in-flight is asserted. OffsiteInventoryList had no test at all before this. TEMPLATE SAFETY - every Restore* key is set UNCONDITIONALLY in the deploy handler, because a template doing index/eq against an undefined key errors at RENDER time: green build, green vet, green suite, 500 on the page. Four render tests, one per branch, because the existing deploy render test only renders AutoFields and never reaches these blocks. RED-PROOFS, mutation asserted applied then reverted to 0: A three template guards dropped (count asserted 3) -> the blank form returned P4 inventorySizeConcurrency = 1 -> "peak in flight was 1", elapsed 282ms = sequential DOCS: CHANGELOG v0.217.0 (MinAgent 0.129.0 unchanged), CONTEXT (the restore's own memory + what is next), controller/README.md (Backup System), REUSE.md (4 new rows), REPORT.md overwritten - the previous REPORT preserved to audits/REPORT-v0.216.0-2026-08-14.md first. NOT fixed here, filed as R-353 and named the next session's first item: a restore whose unit carries no db_dumps and no volume_dumps still reports a bare completion.	2026-08-21 21:29:01 +02:00
admin	90f2545679	fix(disk-health): one physical disk must be evaluated once per run (R-335) ... Found on live hardware two hours after the v0.215.0 deploy, by noticing the release's own positive observable disagreed with its own persisted artefact: the check logged '3 disk(s) evaluated' while disk-health-state.json held two records. demo-hp's c11-scratch and felhom-backup are the same NVMe and share a durable id, so one disk was walked twice per run. Not cosmetic. The loop writes a disk's record before the next entry reads it, so the second copy of an aliased disk consumed the FIRST copy's write as its prior: the disk sustained against ITSELF and reached Hiba on a first sighting, defeating truth-table row 6 — the rule that separates a one-hour benign excursion from a false critical. It would also have emitted two identical events for one drive. Latent on demo-hp only because all counters are zero. Each diskKey is now evaluated once per run. Both entries stay marked seen so neither looks like a disappeared disk, and the card still renders both rows — the dedup is about state and alerts, not display. Red-proof run and reverted: deleting the guard makes the first sighting emit Kind:2 (Hiba-from-sectors) at 8 sectors.	2026-08-14 10:30:22 +02:00
admin	8144a70a72	docs(v0.215.0): CHANGELOG, CONTEXT decisions, README feature, REUSE entries ... - CHANGELOG leads with the severity fix and the live warning-vs-warn proof. - CONTEXT records the settled decisions so they are not re-litigated: Hiba is the label for predicted failure (no fourth word); sustain before count and why; the provenance of 64/55/60; phase 2 owns the new SMART attributes because they are a wire change under G-1; phase 1 state is one record per disk, not a series. - README documents the 14-row ladder, the persisted state, the hourly cadence and the five message shapes. - REUSE pins the severity wire contract on PushEvent — the defect's real home, so the next typo'd severity is caught at the table rather than in production — and records priorFor vs cardPriorFor, which differ by one observation and make the chip disagree with the email if mixed up.	2026-08-14 08:34:31 +02:00
admin	3ed5e3e770	v0.214.0 — the recovery screen stops hedging about a code it can now check (R-311) ... MinAgent: 0.129.0 What was already right: the screen did not bluntly accuse. R-222/R-226 hedged, naming both causes and the kept package, and saying it could not tell them apart. That was honest - and it could not tell them apart because nothing ever looked. Agent v0.129.0 looks, so the hedge becomes an answer. New class RecoveryCodeOpensRetained on HTTP 422, gated by FeatureRetainedRecoveryClass (MinAgent 0.129.0). The gate is SEPARATE from the R-224 one because the two name different agent versions and a box can sit between them, where a 422 is a shape we did not design. ClassifyRecoveryFailure therefore takes both flags; the compiler found every call site. The message says the code is correct, names the supersession date, says the earlier package is kept, and says the CURRENT backups are unaffected - the half a customer will otherwise assume wrong. It promises NO restore: there is no in-product route to a set-aside store (R-312) and the retained package may itself predate the repository-password field. It routes to support, which can do it. The claim guard grew a surface and immediately convicted something. It scanned templates only, while every recovery message is a Go string in a handler - the highest-stakes copy in the product, never scanned. It now scans recovery_handlers.go too, and found a PRE-EXISTING unregistered claim on its first run. Six handler tests asserting which SENTENCE the customer sees; red-proofs asserted applied, including: 422 unconditional makes an agent that never looked read as having looked, and routing 400 to the new class congratulates a mistype.	2026-08-12 18:42:02 +02:00
admin	89712563a0	R-302: the abandon banner promises only what the box can still see is true ... The retrieval clause rendered unconditionally on every page and is false on a reachable state - the same screen where the orphan card says we cannot tell. The condition is a fingerprint PINNED at the decision, not a comparison against the current key. The obvious proxy asks about the wrong key: the set-aside copies were written under an older key the box no longer has, so on a twice-rebuilt box the proxy promises about copies nothing can open. Demonstrated - under the proxy, the replaced-package and legacy cases both flip back to promising. The pin is a recorded assumption and says so: nothing on the box records which key wrote those copies. Empty is not a match. A countdown started before this carries no pin and takes the cautious branch, not a backfill. A sweep of all 36 templates found a fourth instance (backups page, same condition applied) and a fifth (the confirmation screen, correctly left alone - true at the moment of the decision). New retrieval_promise_gate registers each claim with a reason rather than banning a verb: a string ban failed twice, and the honest replacement copy contains the stem.	2026-08-12 15:27:29 +02:00
admin	68f3e12398	R-299: the orphan card's second promise, and a guard that matched one inflection ... The explanation paragraph - the always-visible half of the card - still ended 'a hozzajuk tartozo helyreallitasi koddal kesobb visszaallithatok lehetnek', the same unevaluable claim v0.211.0 removed from the confirm block below it. It survived because the spec called that line accurate, and because the regression guard asserted the SINGULAR form while the card carried the plural, which does not contain that substring. The guard now matches the stem, so any conjugation fails it. The two accurate halves are kept. Also: the guard's failure message sliced rendered HTML at a byte offset and cut Hungarian mid-character; it now slices on rune boundaries.	2026-08-12 13:49:15 +02:00
admin	86a78c6767	R-294/R-295: orphan card stops promising restorability; one name per secret ... The orphan card told a customer their set-aside off-site history may be restorable later with their recovery code. The discriminator lives on the hub and no wire field carries it, so the box rendering that card cannot evaluate the promise. Copy replaced per the spec: state what happens, decline what we cannot know and say why, name a route. The claim page called the same three-word dashboard code two different names depending on branch, one of which collides with the ten-word escrow code. Retired 'Visszaallito kod'; the name is now constant and the sentence changes. Naming only - a test pins that a reset code is still accepted. secret_in_markup_gate no longer convicts Go template comments, which are stripped before render; still convicts a real rendered secret.	2026-08-10 13:53:21 +02:00
admin	b762a37097	R-280: attach list from mounted-but-unregistered filesystems; two-clicks promise made conditional ... After a reinstall the data drive could not be re-attached through any dashboard route: both candidate lists came from the agent's unclaimed-disk scan, and the rebuilt box's drives are claimed. The restore page said it was two clicks while pointing at an empty picker. The attach list now also carries the controller's own mounted-but-unregistered filesystems. initialize is untouched, so the format wizard's system/backup protection is unchanged. The 'two clicks' sentence is conditional on the picker being non-empty, and says something true and actionable when it is not.	2026-08-10 13:41:32 +02:00
admin	c732fe1283	v0.210.0 — R-259 and R-258: two pictures that were not true ... Both are one shape: something the box already knows, drawn as its opposite. R-259 — A DISK WE FAILED TO READ WAS DRAWN AS A HEALTHY EMPTY DISK. readDiskUsage (internal/system/info_linux.go) logged a statfs failure at DEBUG and returned, leaving the caller's TotalGB/UsedGB/AvailGB/Percent at zero — and usageColor(0) is "nominal". The dashboard's most-looked-at meter therefore rendered "0.0 GB / 0.0 GB (0%)" with a 0%-wide bar in the healthy colour. "We could not look" and "there is plenty of room" were the same picture. readDiskUsage now returns whether the measurement succeeded; SystemInfo gains DiskKnown and HDDKnown (HDDConfigured is not a substitute: it says a path was configured, not that reading it worked); and the template draws NO figure, NO percentage and NO meter fill when unknown, saying "A tarhely merete most nem olvashato ki." instead. A healthy box is byte-identical, colour band included. This session rules the convention (felhom.eu CONTEXT.md S-39): an explicit `...Known bool` companion beside the figures, checked in the template — the shape Offbox.StatsKnown already uses, whose own comment says "a 0%-wide bar over an unread store is a picture of emptiness, and a picture is a claim". Pointers and separate error fields are both legitimate Go, but a codebase with three dialects cannot be gated (ROADMAP G-3 was blocked on exactly this). Existing call sites NOT converted. R-258 — THE PER-APP BACKUP TICK WAS GREEN ON PRESENCE, AND RED ONLY ON A GLOBAL CONDITION. buildAppBackupRows set Tier1LastStatus from status.LastDBDump.Success, which is the box's single most recent dump RUN, whichever app it belonged to. An app whose own dump failed showed a tick as long as some other app dumped successfully afterwards; an app with no database took the nil branch and went green on the mere existence of a restore point. appDumpVerdict now reads THIS app's own entries in DBDumpStatus.Results (matched on DumpResult.DB.StackName, failure = non-nil Error). Three states: any failing database -> error; all clean -> ok; no result recorded -> NO verdict and no icon, titled "Errol a mentesrol nincs eredmenyunk." The recovery unit carries no per-run outcome of its own, so green cannot honestly be derived from presence. The global tier1DBStatus label is untouched — it is correct as a global. RECENCY IS DELIBERATELY NOT ADDED. A tick over a three-week-old restore point is a real weakness, but an age threshold means inventing a number and the time is already printed beside the icon. Recorded as an observation, not changed. AN EXISTING TEST WAS ASSERTING THE DEFECT AND WAS CORRECTED, NOT DELETED: TestBuildAppBackupRows_Tier1FromRestorePoints expected "ok" for a status with no LastDBDump at all — green from nothing but a file's existence. It now expects no verdict; its real subject, the Tier1LastRun time, is unchanged. The dashboard test EXTRACTS the meter block from the shipped template rather than copying it: a copied block drifts, and a drifted copy passes while the page it claims to cover has changed — the fixture-is-not-the-wire mistake this project has now hit twice. Six red-proofs across both parts, each with the mutation asserted applied. No new tag on any declared wire — report/builder.go maps into its own types and is untouched; wire_contract_gate.py confirmed green. go build / go vet / go test ./... green (28 packages), controller_gates --fast all OK, both run separately from this commit.	2026-08-08 16:29:52 +02:00
admin	fcffaf573a	v0.209.0 — R-247: the box stops saying a false thing about its own recovery package ... The answer was on the wire and was discarded at the boundary, for the third time. The hub has sent `escrow_stale` in the report ACK since v0.57.0 (json:"escrow_stale,omitempty"). report.EscrowStatus had no field for it, so encoding/json dropped it, and an empty restic_pw_sha256 had exactly one possible reading here: "hash-less supersession". On demo-hp that reading was false in EVERY clause for four days, and the box told the customer so in its own words. The hub HAD the hash and was withholding it because the escrow row carries a stale flag (R-246); there had been no supersession; and the bundle DID cover the password — the hashes matched exactly. Fixed by receiving the field. EscrowStatus.Stale decodes, and reconcileEscrowed tells the two conditions apart: a withheld hash now reports that the hub has flagged the row and is withholding, that this box therefore cannot verify its bundle either way, and that it is NOT established that the bundle fails to cover the password. The genuinely hash-less case keeps its wording. Deliberately NOT changed, and said rather than skipped: the stale verdict itself (the hub's flag is still the hub's verdict; runs still continue), and the customer-facing Hungarian card copy. Clearing the wrong flag is an operator act hub-side (R-246); re-wording the card is UI work with its own review path. This change is the wire and the diagnosis. Found by felhom.eu/scripts/wire_contract_gate.py (G-1), which was built first and seen failing on 40 fields before anything was fixed, and which now refuses any new field of this shape. go build / go vet / go test ./... green, run separately from this commit.	2026-08-08 08:46:29 +02:00
admin	27d1165962	v0.208.0 — R-254: the last two secrets leave the page source, plus a gate against a fourth ... Site one. app_info.html rendered {{.InitialCreds.Password}} into a hidden span — a REAL per-install credential, read live out of the running container, in the response body of every render. The page now carries the non-secret half plus a boolean; the value comes from POST /apps/<slug>/initial-credentials/reveal, which RE-READS the container rather than serving a cached copy (caching it in the handler would put it back in the body one layer in). no-store, CSRF-covered, logged as an act. Both buttons go through it. A reveal that cannot read the value SAYS SO rather than returning an empty string that renders as a blank password. Site two, established before changing. The hidden input is NOT the defect and was left alone: it fires only pre-deploy, and README §318 documents why the value must round-trip — the customer notes the generated secrets down and submitting them back is what makes the saved value the same one they saw. The defect was the neighbouring READONLY input, which on an ALREADY-DEPLOYED app rendered the secret into a page with nothing to submit. Fixed by POST /stacks/<name>/auto-field/reveal, authorised by requiring a type:secret auto-field of that stack. Both directions pinned. The premise that this contradicted a repo rule does not hold: the rule is CONTEXT.md:2070 'Password fields require explicit input — prevents accidental empty-password deployments', about EMPTINESS. No line in the repo says 'no silent auto-fill'. The gate. scripts/secret_in_markup_gate.py, registered in controller_gates.py, convicts any template expression that names a secret unless allowlisted with a reason. Its limits are MEASURED and in its docstring: it catches a launder through a local variable (the assignment names the secret) but is blind to a secret arriving under a neutral page-data key — verified both ways. That is the shape of site two, which this gate would NOT have caught. The runtime body assertion covers all shapes but only 4 of 27 page templates; the other 23 are R-255, filed rather than glossed. Two nets, different holes, both named. Correction to v0.207.0's report: HTML comments do NOT ship in the response body here — html/template strips them, text/template does not. Measured. A red-proof planting a secret in a comment therefore correctly does not fail.	2026-08-07 21:20:26 +02:00
admin	8dbbc98ff2	v0.207.0 — R-249: the retrieval passphrase leaves the page body; R-252/R-253: two refusals learn to say what to do ... R-249. settings_security.html rendered the passphrase into a display:none span behind a Megjelenit button. That toggle stops a browser DRAWING the value and nothing else — the plaintext was in the response body of every render, so a curl of the page returned it. Found by exactly that: it landed in a session transcript while driving the documented rebuild path. The codebase already stated this rule for the recovery code and this page did not follow it (escrow_handlers.go: 'reveal (claim XHR only — R is NEVER templated server-side into HTML)'). The page now carries only HasRetrievalPassword; the value comes from POST /settings/retrieval-password/reveal — CSRF-covered because POST, no-store, and LOGGED as an act, which reading it off the markup never was. The tests assert the RAW RESPONSE BODY. Every test that asked what the customer sees passed while the bytes carried the secret; that is why this survived. Census: the render-then-hide pattern appears twice more — app_info.html (a real per-install app password in a hidden span) and deploy.html. Filed as R-254, NOT fixed here. R-252. A rebuilt box keeps its drives but loses their REGISTRATION. The restore page now states that before the customer presses anything, says the backups and drives are both still there, and links to Tarhely > Meghajtok. Page and resolver ask ONE question — HasRestoreDestination() reads the same GetSchedulableStoragePaths() the scratch resolver reads. R-253. The list promised 'a visszaallitas elobb ujratelepiti' three lines above a refusal that fired BECAUSE the app was not installed. The promise was the wrong half: reconstitution writes to the app's own GetStackHDDPath, which exists only once the CUSTOMER has chosen a drive at deploy time. Auto-reinstalling would mean the product making that choice for them. Copy now says to install first and routes to /stacks/<app>/deploy. Both notices are conditional — a healthy box renders as before, pinned by a test that fails if either becomes unconditional.	2026-08-07 18:04:26 +02:00
admin	0a9158d53e	docs for v0.206.0: CHANGELOG, CONTEXT, REUSE, README ... CHANGELOG v0.206.0 with the ruling that reversed the fix, the three changes, the SEC 7.2 staleness decision, Q7's closed trap, and the two bugs the tests caught rather than review. CONTEXT carries the three rules this session established, in the form the next session needs them: - a box does not create a repository key while the hub holds a sealed package for it; - the fact that answers a question must be kept where the question is asked; - fix the state, do not remember that it is wrong. REUSE gains four rows, each carrying the trap rather than just the signature: the mint guard is a CONJUNCTION and t.Enabled is load-bearing in the derived predicate; the discriminator ships INERT unless wired in main.go's confirmer literal; the countdown removes BOTH halves or neither and must be driven by an injected clock; and the epoch must be synced FIRST and unconditionally or the falling edge is lost. README documents the three customer-visible changes and the operator levers. No version literal was edited: the controller version is ldflags-only.	2026-08-07 12:12:35 +02:00
admin	c6b69d888e	v0.205.0 — a run that skipped an app the customer selected is not successful (R-234) ... THE VERDICT. The R-203 block already said "a warning beside a success is read as a success" and applied it to ONE of the two shapes it describes: an app missing a declared mandatory FOLDER made the run incomplete, while an app skipped ENTIRELY still reported ok. Both do now. Which skips count, decided by measurement: selected+deployed with no recovery unit YES; selected but NOT deployed no (named, with what to do — a box left amber by an app somebody removed is a status nobody reads); disconnected/decommissioned drive no (own signal); nothing selected no. LastSuccess and SnapshotCount still record what WAS captured. THE FILED MECHANISM WAS NOT THE MEASURED CAUSE, and saying so is the point. §3 stated that toggling an app on leaves it without a bundle so the first run skips it. Measured on demo-hp: the run's own pre-dump phase calls captureAllRecoveryUnits for every DEPLOYED stack, through admitApp, before the push — a unit moved aside was RECREATED and the run reported ok. That state does not survive a run. What actually produced the 2026-08-06 sequence: the manual run was dropped by the single-flight while an earlier run was still going. runOffboxBackup returned nil, the handler had already answered "A tavoli mentes elindult", and the card then showed the PREVIOUS run's green verdict — read as covering the app just selected. The decision is now taken synchronously in the handler and a dropped request says so. The nightly path still returns nil on purpose: nobody asked, and it retries. §7.3 measured before deciding: CaptureRecoveryUnit writes a few KB of compose + manifest, only ENUMERATES dumps rather than creating them, is idempotent and does NOT stop the app — and already runs inside the off-site run. So there is no wait to remove for a deployed app and NOTHING was built. 28 packages ok, 9/9 gates. Four red-proofs, each asserted to have applied. Fixture note: the shared provider's ListDeployedStacks returned nil, so Scenario A first passed for the wrong reason; fixed with an opt-in deployed set that defaults to nil.	2026-08-06 21:58:21 +02:00
admin	53e9bf0224	v0.204.0 — the restore list is keyed on the store (R-237); the size gate stops refusing in silence (R-238) ... R-237: /backups/restore listed apps that are CURRENTLY DEPLOYED and CURRENTLY TOGGLED ON for future off-site backups. A rebuilt box has neither, so a household that had just lost everything was shown nothing to restore while the repository held their snapshots — measured live on the R-201 re-walk. To restore an app you had to select it, to select it you had to have installed it, and to know what to install you had to see the backup you could not see. The store is now the source of the list (offsite_restore_list.go), built on the existing R-193 OffsiteInventoryList. Installed-ness became a property OF a row, never a filter on it. Every case is answered rather than hidden: a snapshot for an app that is not installed is offered and says it will reinstall first; an installed app with no snapshot is shown as having nothing; an unreadable store renders as UNKNOWN (R-225's rule, one screen over) AND keeps the action, because "we could not look" is not "there is nothing"; no-target is its own state. The felhom-offbox and _shares marker tags are excluded from the app list. R-238 classified as a HARNESS ARTIFACT: mode=full without confirm=1 is step 1 of a deliberate two-step — it starts no job by design and redirects carrying &full_prep=<app>, which deriveWizardStep requires to reveal the commit. A driver that did not carry it forward landed back on the intent step. The operator's browser run completed the same restore. The wizard's precedence rules were NOT re-keyed: a stale ?full_prep= must never resurrect a commit button mid-restore. The residue WAS real and is fixed: neither branch of that step wrote anything to the log, so a refusal — including by the headroom gate — left no trace on the box. Both branches now log, and so does the concurrent-op refusal. resolveWizardApp is removed: it was dead once the gate moved, and its test pinned the defect's behaviour (an untoggled app refused), which would have read as policy. 28 packages ok, 9/9 gates OK. Three red-proofs, each asserted to have applied.	2026-08-06 16:44:05 +02:00
admin	9dc26459ea	v0.203.0: the box collects what the hub staged for it (R-218 consume half) + R-220's message ... R-218's declaration half shipped in v0.201.0 and works. Its consume half never existed. Reconcile ran exactly twice per process — at start-up and when the recovery screen drives it — and BOTH fire before the hub has anything staged, because the hub stages in RESPONSE to the declaration those runs precede. Measured on the R-201 re-walk: unlock reconcile 11:43:07, hub staged 11:44:57 saying 'next cycle', a full report cycle ran 11:55:46, still unconsumed at 12:06. A guest command line applied it in 18 seconds — everything correct except the trigger. Bridge.RetryIfDeclared re-runs the SAME reconcile on a 5-minute tick, driven from the box's own published declaration (OffboxReportStatus().State) — the very statement the hub acts on, so the two cannot disagree. Poll, not an ACK flag, decided on the promise: the no-target message says 'amint megvannak' (no deadline) and the card says 'within a day'. Five minutes is inside both by a wide margin and needs no hub change. It stops by construction — a healthy box does no work and logs nothing — and the settle gate is deliberately kept via ReconcileWhenSettled. The marker was investigated and left alone: applied_marker lives in the guest's DataDir, which a rebuild destroys, so it cannot suppress a legitimate re-run. R-220's customer half: the refusal no longer tells the customer to choose from a list that may be empty. It names the rebuild, points at the Meghajtók page, and promises no outcome. Red-proofs: remove the retry -> credential uncollected (the dead end reproduced); drop the stop condition -> a healthy box hammers the hub; call Reconcile instead of ReconcileWhenSettled -> settle gate bypassed; restore the old sentence -> the impossible action returns. 28 packages ok, vet clean, all controller gates OK.	2026-08-06 12:56:12 +02:00
admin	66d80efb9f	docs: R-168 is CLOSED — the "CI is still owed" sentence was stale (R-229 part 2) ... Corrected in all four instruction files across all four repos. Found while confirming this session own push by run ID, which is precisely the check that catches it. In felhom-agent/CLAUDE.md the sentence contradicted the same file release section, which already said R-168 mails the failure -- a contradiction inside one instruction file, the exact class the R-229 work exists to find. REPORT.md deliberately NOT overwritten in the sibling repos: a one-line docs correction must not destroy the record of their last real implementation.	2026-08-06 11:02:55 +02:00
admin	7db42c5fec	docs: CLAUDE.md becomes a core plus path-scoped rules (R-229) ... 215 lines -> 110 (92 effective; block-level HTML comments are stripped before injection and never reach the model, verified empirically on Claude Code 2.1.222 with a control and a treatment run). Four new .claude/rules/*.md, each with a paths: glob list so it loads only when a matching file is read: gates, ui-hungarian, backup-paths, agent-coupling. The ## Layout tree was deleted as derivable; REUSE.md already owns the per-package seams its annotations stood in for. The host/access table was deleted in favour of a pointer to documentation/operations/nodes.md -- it carried three defects at once: demo-felhom given as the LAN fallback address as if it were the route, a pinned "agent 0.93.0" against the project's own no-versions-in-docs rule, and the claim that no drill VM was provisioned on demo-hp. Measured live: qm list shows VM 300 drill-r50. felhom-agent/CLAUDE.md was right; this file was wrong. Kept verbatim: the seven session-critical invariants, the F9 live-validation fence, the end-of-session checklist. controller_gates.py registers the shared instructions gate (felhom.eu/scripts/, never copied here; an absent sibling clone FAILS). Docs only -- no Go, no version bump, no image, no deploy. Ledger: felhom.eu/documentation/audits/LEDGER-instruction-trim-2026-08-06.md Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JJc8sAGRWmavP3rMtdpkr2	2026-08-06 09:38:27 +02:00
admin	7534ea203d	CHANGELOG: controller v0.202.0 (MinAgent 0.126.0) ... Declares the coupling: an agent below 0.126.0 answers 400 for both a fetch failure and a wrong code, so FeatureRecoveryFailureClass withholds the refusal reading and the 400 degrades to the neutral message. The gate blocks nothing — it only decides whether the customer may be told to check their typing.	2026-08-06 08:19:07 +02:00
admin	a3499d1807	v0.201.0 — a correct recovery code is never called wrong again (CAMPAIGN-11) — MinAgent 0.125.0 ... R-216: the offsite key recovery is a coupled feature and now says so. featureProbes + featureMinAgent 0.125.0 + a Supports gate at the unlock entry point, FAILING CLOSED — an agent that cannot answer is named as such instead of the customer's code being blamed. Measured live: a 404 from agent 0.120.0 came back as "we did not accept your recovery code, check that all ten words", in 0.134 s, against a perfect code. R-218: delete the repo-password short-circuit in needsOffsiteCredential. The declaration stops when the TIER WORKS, not when a key exists — installing a key is the recovery screen's whole job, so succeeding at recovery was switching off the mechanism that would have delivered the coordinates to use it. R-219: the unlock finishes the job — place the key, bring the tier up, then list. Without it the promised listing could never render on the shape the screen exists for. R-217: an unreadable store no longer claims to have opened with unattributable content (the OffsiteInventory{} zero value). Opened / empty / unreadable are three states. R-222: a code that is right about a RETAINED earlier package is named, not blamed. States what the hub knows and promises nothing — no read path exists. R-215: GET /recovery is gated on the same predicate as the interception. Five red-proofs, each demonstrated failing and restored.	2026-08-05 17:48:08 +02:00
admin	62b85ecf13	CHANGELOG: controller v0.200.0 (R-193, the recovery screen)	2026-08-05 12:46:13 +02:00
admin	992803c10b	CHANGELOG: controller v0.199.0 (R-204 item 4, box half)	2026-08-05 10:49:20 +02:00
admin	33fcc502e4	CHANGELOG: controller v0.198.0 (R-204 items 1 and 3)	2026-08-05 07:17:32 +02:00
admin	58c703bd44	R-203 Part 2: a run that missed a MANDATORY directory is not a successful run (v0.197.0) ... The gap was already detected and warned about, in Hungarian, naming the app and the folders -- that warning is what stopped the R-201 drill. The defect was that the run still reported `ok` beside it, and a warning standing beside a success is read as a success. last_status gains "incomplete": minted, because "ok" | "error" | "running" had nothing meaning "it ran, and this app is not fully protected". NOT "error" -- the rest of the run worked and what was captured is real, so SnapshotCount and the LastSuccess anchor still record it. Half a backup is not no backup. The gaps are now recorded STRUCTURALLY (offboxRunResult.mandatoryGaps), not only as prose, so the verdict has something to act on. It reaches the operator through the EXISTING per-run digest (backup_run_failures) rather than a new event type -- a new type is a two-repo change and the hub drops anything outside allowedEventTypes. The stat-filter gains the ClassMandatory check Tier 2 already had. It is a NO-OP today (TierOffsite admits mandatory only), so no customer-visible warning disappears -- demonstrated by widening the tier filter alone and watching the check hold the line. ANTICIPATED: calibre-web on demo-hp has exactly this gap, so its off-site status becomes incomplete the moment this ships. That is correct and is the point. Red-proofs: my first Scenario-C proof PASSED because the test only reached offboxCaptureSet while the mutation lives in runOffboxInternal -- a mutation the test cannot observe is not a red-proof, and the fix was the test. The run-level test now fails under both mutations (unreachable gap recording; unconditional ok).	2026-08-04 18:32:56 +02:00
admin	1b1366bb6e	controller v0.196.0: the recovered key installs itself (R-200 plumbing half) -- MinAgent 0.125.0 ... --recover-offsite-install is the sibling of --recover-offsite-check: same fetch/unseal path through the agent, same STDIN discipline for R, but it PLACES the recovered repository password via InjectOffboxPassword so a rebuilt box reopens the history it inherited. Doing this by hand would put the offsite DATA key through a terminal, a clipboard and shell history. In-process the value goes agent -> this process -> the 0600 file and is rendered nowhere. The confirmation is a SECOND invocation: without --confirm-install it prints both hashes and writes nothing, so the operator sees the comparison before any write is possible. Three outcomes, named distinctly: installed (no local password -- the rebuilt-box shape), unchanged (identical key already present, nothing written), refused (a DIFFERENT key present; installing would clobber the key the current repository is encrypted under, and no force option is offered). Exit 2 for the refusal, distinct from 1 for a failed step. Red-proof: removing the confirmation gate makes the dry run write, failing the test. The R-persistence test carries a positive control -- a planted copy is found, then removed and not found -- because an absence check is worth only what its sensitivity is.	2026-08-04 14:27:38 +02:00
admin	9640e51321	controller v0.195.0: prove the offsite key comes back (R-200 plumbing half) -- MinAgent 0.125.0 ... --recover-offsite-check is a docker exec diagnostic in the shape of --print-reset-code: it reads the customer's recovery code from STDIN, asks the agent to fetch this host's sealed bundle and open it, and reports whether the recovered key matches the one on disk BY SHA256. Two hashes and a verdict; never a password, never R, never a blob. R comes from stdin and not a flag because a flag value is visible in ps, in shell history, in a container's command line and in any transcript of the session that ran it. IT COMPARES; IT DOES NOT INSTALL. The recovered password is never written to offbox/repo_password -- installing changes a live box on a path nobody has walked, and that link is next session's, with the drill around it. A test asserts the data dir is byte-unchanged after a check; its red-proof (adding the install call) fails it. Exit codes: 0 match, 2 clean MISMATCH, 1 a step failed -- "it failed" and "it worked and disagreed" must never share a status. A box with no local password reports distinctly: that is the rebuilt-box shape, where the next step is to install rather than compare. Nothing customer-reachable ships here: no card, no form, no preview.	2026-08-04 13:42:50 +02:00
admin	88897a224e	v0.194.0 — one operator email per backup run, and nothing dropped without a trace (R-182) ... MEASURED, not supposed. On 2026-08-03 nine per-app recovery_unit_capture_failed events reached the hub and TWO operator emails went out. The hub's operator cooldown key is customerID:eventType(+tier) and that event carries `app` but no `tier`, so the key held no app identifier: the first refused app took the hour's slot and every other app's failure was discarded BEFORE anything was written down, leaving no row on any channel. The obvious fix — put `app` in the key — was ruled against: on a full disk it produces one email per app, the volume problem wearing the correctness problem's clothes. internal/backup/runsummary.go: a per-run collector with exactly admissionSet's lifetime, fed by all three write legs, emitting backup_run_failures ONCE at the end and only when something failed. A clean run emits nothing. The per-app event stays and becomes the RECORD — the hub routes it record-only, stored and logged every time, never competing for an email slot. The record and the notification are now different things. Deliberate skips (disconnected, decommissioned) are excluded: they have their own alert, and a nightly email about an unplugged drive is one the operator learns to ignore. A manual run always reports: the digest carries a unique run_id the cooldown cannot collapse. Someone pressing the button is actively trying to get a backup. THE PERIODIC SWEEP GETS A DIGEST TOO. With the per-app event now record-only, a capture failure found between runs would be recorded and never notified — a new silence introduced while closing one. That path emits a digest with NO run_id, so the ordinary 1-hour cooldown caps it exactly as before while the mail now lists every failing app instead of whichever was first. A refusal is recorded ONCE, where the verdict is taken, not at the three legs that consult it — R-181's contract is one verdict per app per run. Noting it per leg listed one refused app three times and produced "2 of 1 apps failed". Found by the digest's own test, not in review. Silence is safe because the hub's deadline check raises expected_backup_missed from report freshness, independently of any mail this box sends (monitor/deadline.go:396,417). Confirmed, not assumed. 7 new tests, 4 red-proofs. The main.go seam walk did NOT fail on its first attempt — the AST test walked the backup package and not main.go; the test was fixed and the mutation re-run rather than the pass recorded.	2026-08-03 13:46:14 +02:00
admin	6c43bf6156	v0.193.1 — the refusal's size estimate is rendered in bytes, not "0.00 GiB" (R-181 follow-on) ... Found by v0.193.0's own live proof run. The estimate was printed fixed to two decimal GiB, so every app under ~10 MB rendered as "estimated 0.00 GiB write" — which reads as "no estimate was available" and is the opposite of what happened. Observed live on demo-hp 08:59:46: opengist's real 178 KB estimate printed as 0.00 GiB. Shipped in the same session because it is the same defect class R-181 is about: a message an operator cannot rely on is worse than no message. The arithmetic is unchanged and still in GiB — the reserve's own unit, so the comparison against FloorFreeGiB reads directly. Only the rendering moved to humanizeBytes. estimatedWriteGiB -> estimatedWriteBytes, with the GiB conversion done once at the point of comparison.	2026-08-03 11:05:02 +02:00
admin	fef07c3923	v0.193.0 — the reserve guards the write that fills the disk, and its promise is true (R-181) ... B2's capture floor (v0.192.0) was consulted in exactly ONE place — captureAllRecoveryUnits, which writes a few KB. The two legs that write the BULK into the same backups/primary/<app> tree, the DB dump and the volume dump, ran FIRST and unguarded. Measured live on demo-hp 2026-08-03 06:40:03: opengist's volume dump wrote 2.0 GB with no check, free fell to 1.0 GB, and the floor then refused the cheap write it had already lost the argument to. Its refusal message claimed "the previous unit is untouched" — measured false: that app's tar had gone 182,272 B -> 2,147,666,432 B under a stale manifest. Sixth entry in CLAUDE.md's table of shipped guarantees the code did not provide. Fix: ONE admission verdict per app per run (internal/backup/admission.go), taken before that app's FIRST write and covering all three legs — they write under one per-app root, which is why one verdict can honestly cover them. - Lazy, at the app's first write, NOT once at run start: app A's dump can put app B under the reserve, so a run-start verdict reads a disk that no longer exists. - Remembered for the run, never re-decided between an app's own legs — that is the split this closes. Reset per run. - Placed ahead of DumpAppVolumesSafe, which stops the stack as its first act, so a refused app is never bounced. After the volume-less check, which has no write. - Exactly one operator alert per refused app per run. - Leg order unchanged: volume dumps still precede the capture. The floor is now SIZE-AWARE: it asks whether THIS app's write would cross the reserve, not only whether the filesystem is already below it — which is how an app was admitted at 96% and then allowed to write 2 GB. Estimate = the app's previous .sql + .tar on disk. No history -> headroom-only, deliberately, and the alert says so. A container-based du per volume was MEASURED and rejected: 66 timed runs on demo-hp guest 9201, median ~355 ms/volume (341-404) on volumes holding tens of KB — container start-up, not the walk. Decisive on top: docker run needs the writable layer, so it can fail under exactly the pressure the reserve handles. The message was NOT weakened; the behaviour was moved so the wording became true. It now also names which term bound. Every claim is checked against a sha256 fingerprint of the tree it describes, never against the log line. Still refuses and never deletes: nothing here is generational. 11 new tests through the production functions. The DB leg cannot run without Docker, so its gate is pinned by an AST walk of backup.go asserting admitApp precedes DumpOne (strings.Contains is insufficient — a commented-out call still contains the string). 4 red-proofs demonstrated failing then restored.	2026-08-03 10:53:48 +02:00
admin	4be6467b50	v0.192.0 — the capture floor replaces the bulkhead (R-165, decision B2) ... Ships BEFORE the disk-layout merge it exists for, and is harmless on a box that never gets it. The mp1 partition was a BULKHEAD as well as a ceiling: it kept a runaway capture from filling the space the container runtime needs, because /var/lib/docker was a different filesystem. After the merge it is the same one, and a full Docker data-root is a stopped box. The floor sits in captureAllRecoveryUnits, checked BEFORE anything is written: below the reserve, that ONE app's capture is refused, its previous unit is left byte-identical, the R-158 alert fires with the space figures, and the loop continues. Two terms whichever binds first (97% used / 1 GiB free) in fillwatch's shape, deliberately BEYOND its critical band (95% / 2 GiB) so the customer is always warned before a refusal can happen — a floor that fires before its own warning is a silent failure wearing a threshold. Headroom, never unit size: a per-unit cap would be R-163 rebuilt inside one volume. Refuses, never deletes: nothing here is generational, so pruning could only destroy a different app's only local copy; pruneStalePrimaryDirs is an orphan sweep, not retention, and must not be repurposed. Tests 1184 -> 1191. One fixture strengthened mid-red-proof: the "old 20 G ceiling is gone" test sat at exactly 20 GB and survived a literal UsedGB > 20 cap — hollow. Now 120 GB, and the mutation fails it.	2026-08-03 06:30:19 +02:00
admin	9a3c4855d7	v0.191.2 — a quiet fill check now says so (R-167) ... Earned during v0.191.1's own live validation. After the customer had been warned, a restart produced ZERO fillwatch lines — equally consistent with 'ran and chose silence' and 'never ran'. Proving the checker was alive needed a deliberate crossing into the critical band. For an edge-triggered check the quiet run IS the healthy steady state, so that ambiguity is permanent rather than rare. Check now logs a per-RUN summary on every run, counting unreadable separately from healthy so a drive that has quietly gone unreadable cannot read as 'all fine'.	2026-08-02 23:37:04 +02:00
admin	5adae4dad9	v0.191.1 — the fill check also runs at startup (R-167) ... Found live on 9201: neither sched.Daily nor sched.Every fires on registration, so a box booting with a filesystem already over the line would stay silent for up to 24h — the R-100 shape, and the same gap the hub's own checkers avoid by leaving already-breached keys unseeded at init. The watcher now runs once 90s after startup as well. Safe because the check is edge-triggered against persisted state: an already-warned filesystem stays silent. The delay lets mounts settle so a drive still returning reads as unreadable and is skipped rather than warned about. Pinned by an AST assertion — the schedule registration alone no longer satisfies the test.	2026-08-02 23:27:35 +02:00
admin	cf48214f6c	v0.191.0 — warn before the wall comes down (R-167, R-158, R-174) ... R-167: new internal/fillwatch warns the CUSTOMER before a filesystem fills. It emits the PRE-EXISTING disk_warning/disk_critical pair, which was allowlisted, copy'd, default-enabled and checkbox'd with no producer in any repo — the sixth "built but never wired" instance here. Two threshold terms (85% or 5 GiB free; critical 95%/2 GiB) because a percentage alone lies at both ends of this fleet's size range. Edge-triggered on escalation only, state persisted, hysteresis dead zone at 75%/7 GiB pinned by a test. A nil usage read is never a warning and never clears one. Per filesystem, never per app. Daily 03:30, before the nightly app-data legs. R-158: new unitNotify seam fires per app when a Tier-1 recovery-unit capture fails, loop continuing, carrying the target filesystem's used/free bytes. Operator-tier (recovery_unit_capture_failed) — deliberately NOT backup_failed, which is customer-enabled and would email the customer about a failure they cannot act on. D-c overrides R-158's own proposal here. R-174: the app-stop guard no longer starts apps onto MISSING drives — a regression in v0.189.0 code, found by review and closed the same session. SetStarter got the raw stack manager, whose StartStack has no drive gate, and Recover runs at startup. R-171 one path over. bootDriveGate could not be reused whole (its holder #2 is the guard's own marker, and holders #1/#2 read vars assigned after Recover runs), so holder #3 is extracted into a shared driveStartGate with a test pinning the delegation. ErrStartRefused splits a refusal from a failure: both keep the marker, only Failed alarms, because routing a deliberate hold into NotifyBackupFailed is the same false alarm. Tests 1157 -> 1184. All red-proofs demonstrated failing and restored.	2026-08-02 23:18:51 +02:00
admin	4bad6e06c9	CHANGELOG: the boot window's sample refreshes first	2026-08-02 20:17:24 +02:00
admin	582135f861	v0.190.0 — the boot settle window, both gates on intent, and R-171 ... R-171 (a regression v0.189.0 introduced, CONFIRMED on hardware before any fix was written). Replacing isBootOrphan's container-count term with recorded intent made a drive-gate-stopped app read as a boot orphan: the gate stops apps with `compose down` (zero containers) and never touches desired_state, because it is not the customer. Observed on 9201 with the drive held unmounted — the sweep found and started it, burned both attempts, and handed it to the dead-app alarm. The write hazard did not materialise (the unbound mountpoint is host-root-owned and the guest is unprivileged) but that protection is accidental and untested. New consumer-side seam bootrecon.StartGate, fail-safe (cannot determine ⇒ do not start), wired in main.go. The rule is not new: the API's startGatedByMissingDrive already refuses this; the sweep bypassed it. R-157 mechanism A. The sweep looked once at T+5s, deriving candidates from a fleet docker was still restoring — three of six hard resets. Now a settle-then- sweep window: sample every 5s, settled after 3 identical samples, sweep ONCE at the end; ends on settled or a 50s budget, and the log says which. The budget is 50s because settle+budget+one retry must stay under the 90s dead-app grace — a test rejected 60s at 95s. A window that overruns emits a LATE RECOVERY warn rather than the grace being widened to hide it. Widening the window made two more holders reachable, so the one gate covers all three: an absent drive, a quiesce, and an in-flight app-data operation — reusing quiesce.SuppressedStacks() and a new read-only AppStopGuard.HeldStacks(). R-170. shouldRecreateOnBoot now reads desired_state with the identical three-way table; absent keeps the old hasContainers behaviour exactly. Its comment argued for the container count and was rewritten. presentStable is untouched. The two gates' agreement is pinned from both sides against one fixture table. 27/27 packages green; 6 red-proofs observed FAIL then restored.	2026-08-02 19:56:20 +02:00
admin	dbcb306fcf	v0.189.0 — desired state + the app-stop crash marker (R-166 / D-b) ... The box stops inferring the customer's intent from a container count and reads what they actually asked for. Part 1 — desired state. AppConfig gains a tri-state `desired_state` (""/running/stopped), written ONLY by the customer's own action: the API action switch, DeployStack, UpdateOptionalConfig's redeploy branch, and the .fab import. Intent is written BEFORE the act and a failed write REFUSES the act. StartStack/StopStack are deliberately not writers — 14 callers, only 2 are the customer. bootrecon.isBootOrphan now reads intent instead of len(Containers)>0, which closes R-157 mechanism B (a power cut or interrupted deploy left an app with zero containers, read as a deliberate stop, and stranded silently). ABSENT MEANS UNKNOWN, NEVER "running": every pre-v0.189.0 app.yaml reads absent, so the legacy fallback is byte-identical to the old rule. A running-only startup backfill converges the unambiguous cases; `stopped` is never inferred. Part 2 — backup.AppStopGuard, a persisted marker over every stop→work→start window (volume dump, offbox reconstitute, .fab export). Its own file, never quiesce's. Written before the stop, cleared only after a restart that succeeded, kept when one fails. Recover() completes before the boot reconciler is launched and returns its outcome, which main.go reports on the existing backup_failed event once the notifier exists. A defer is not the mechanism — a SIGKILL runs none (Campaign 8 fault 10). Also: SaveAppConfig rebuilt AppConfig field-by-field (the R-100 shape) and would have dropped desired_state on every save across nine call sites. Replaced with copy-and-overlay. Measured: app.yaml does not round-trip unknown YAML keys. No hub change, no agent coupling, no user-visible string. 27/27 packages green; 7 red-proofs observed FAIL then restored.	2026-08-02 18:40:17 +02:00
admin	e7c44c0e0f	docs: CHANGELOG + REPORT for the CI workflow (no version bump)	2026-08-02 16:35:42 +02:00