981 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
admin	e43b5ec07d	v0.231.0 - the box proves its own off-site copy still holds something (R-87) ... R-87 re-scoped by its own spike and built as Option C. MinAgent 0.129.0 unchanged. THE QUESTION NOTHING ASKED. The weekly check proves the stored bytes are the bytes we stored; it cannot tell us we stored the WRONG thing. A hollow recovery unit backs up cleanly, checks cleanly at 100 percent depth, restores cleanly and gives the customer nothing back - measured on demo-hp 2026-08-31, 120082104 B to 7036 B in one nightly run recorded as a success (R-403). No tier and no cadence asked it. Now offsite-proof does, nightly, on one app. IT DOES NOT prove a restore puts data back into a running app. That stays drill work and 07 section 8 matrix row 4 is NOT moved. THE ACCEPTANCE RULE HAS TWO PARTS AND THE OBVIOUS ONE IS A TRAP. "Check the unit against its own packing list" PASSES a hollow unit, because a hollow unit declares nothing. So: (1) everything declared is present, AND (2) the manifest declares what the app is supposed to have. Part 2 is the whole value. RED-PROOFED: the naive rule makes the hollow-unit test read verdict "pass". THE EXPECTATION COMES FROM INSIDE THE UNIT, never the live box - the snapshot may predate the app's shape, and GetDockerVolumes describes the running app. Database half is DBServiceNames, the same discriminator RestoreFromRecoveryUnit uses. Volume half is ParseComposeNamedVolumes as an EXISTENCE check, not a name match: tars are <project>_<volume>.tar and ResolveDockerVolumeNames derives the project from the compose file's parent dir, which inside a unit is the literal string "compose". Measured on all eight real units on demo-hp the counts match exactly and the naming held every time - but "held on eight" is not "derivable" (R-355). Half a rule that is true beats a whole rule that is invented. THREE OUTCOMES: pass, fail (readable and empty), cannot judge. An app that legitimately has neither a database nor volumes PASSES. RED-PROOFED: alarming on any empty unit makes that test read verdict "fail". IT NEVER WRITES TO THE REPOSITORY and that is asserted on the ARGV as a non-effect: --no-lock, no unlockStale, and m.runner() rather than resticStep so the unlock --remove-all escalation is unreachable. RED-PROOFED: routing it the customer path's way makes the test fail on "unlock" appearing in the argv. IT TAKES acquireRunning ITSELF and skips rather than waits, because RestoreOffboxScratch does not take it (R-408) while offbox_integrity.go states that invariant as universal. DUE-NESS IS PER SNAPSHOT (R-86's model), never per clock. RED-PROOFED: recording a timestamp fails the stored-value test AND breaks the rotation - night 2 re-picks night 1's app. ITS SCRATCH IS A SEPARATE ROOT (backups/offsite-proof) and that is a safety decision, not tidiness: the job deletes its copy on every path, and sharing backups/offsite-restore/<app> would mean a nightly background job deleting the verification copy a CUSTOMER is looking at. It is also invisible to placement, so a proof copy can never be pushed into a live app. SHARED RATHER THAN FORKED: offboxScratchDirIn parameterises the scratch resolver on its ROOT builder, and unitOnlyHeadroom extracts the free-space gate, so the customer path and the proof refuse at the same floor with the same Hungarian sentence. RestoreOffboxScratch's behaviour is unchanged. NEW EVENT offsite_proof_empty, severity error, operator-only - deliberately NOT backup_integrity_failed, whose hub template says the store is DAMAGED. Here the store is sound and the content is absent: different cause, different action. The hub half shipped FIRST, in felhom.eu 1aeaa30 (hub v0.110.0, live and verified), because an unallowlisted type is 400'd and vanishes. 33 new tests, all groups green; full suite 1689 tests, 28 packages, rc=0. All 13 controller gates OK. Five red-proofs run and recorded in REPORT.md. A golden carrying 0.231.0 is OWED - the fleet is on 0.230.0. Viktor's call (R-242).	2026-08-31 20:55:34 +02:00
admin	2d802d75e8	docs: record the two documentation commit hashes in REPORT.md section 4	2026-08-31 14:39:41 +02:00
admin	1cfdde968f	docs(v0.230.0): R-403 — CHANGELOG, CONTEXT rulings, README, REPORT ... CHANGELOG v0.230.0, leading with the measurement rather than the fix: 120 082 104 B -> 7 036 B on the shipped v0.229.0, reproduced before anything was built. CONTEXT records three rulings: hollowness is a MANIFEST question and never a size question; the guard fences one shape and NOT shrinking, because the derived-copy rebuild is a design decision; and the rehydrate happens inside the restore because a follow-up job races the 5-minute capture. Plus the shape the live run taught: a warning that fires on everything costs the same as the comforting lie it replaces. README documents the refusal, what each surface says, and why the capture job is deliberately not guarded. REPORT leads with Part 1's result, carries the six red-proofs, the per-row Scenario D table with its seven-app control, and eight observations including R-404 filed-not-acted-on and three mistakes of mine recorded rather than tidied away.	2026-08-31 14:39:13 +02:00
admin	b48a7fa326	R-403: the restore OUTCOME names the package's date, not the run's ... Live on demo-hp the confirm said 11:43 (the preserved package) and the outcome said 14:23 (the copy's newest run) for the same restore. A customer reading both cannot tell which one they had, and one of the two is the flattering sentence. Part 2.3's rule is 'not a plain green success ANYWHERE', and the outcome is an anywhere. tier2UnitSourceMsg now asks UnitRestoreDate, the same resolver the confirm uses, so the two cannot disagree. TestR403_OutcomeNamesThePackageDateNotTheRunDate pins it, with a negative control for the ordinary case.	2026-08-31 14:32:29 +02:00
admin	5429d651ee	R-403 fix, caught by the live run: the stale flag fired for every healthy app ... UnitRestoreDate also compared the package's date against the run's and flagged 'older'. A recovery unit is ALWAYS captured shortly before the run that mirrors it, so that comparison is true for every healthy app. Measured on demo-hp: bookstack, kimai, opengist and privatebin all had src and dest manifests at 12:03:49Z against a run at 12:14:24Z - perfectly healthy, and all four would have been told their package was stale. A warning that fires on everything is a warning nobody reads, which costs the same as the comforting lie it was meant to replace. The second return is now UnitLegPreserved and nothing else. TestR403_AHealthyAppIsNeverCalledStale pins it; red-proof: reinstate the comparison -> it fails.	2026-08-31 14:22:32 +02:00
admin	2358e561b7	R-403: a poorer copy must never delete a richer one ... MEASURED FIRST, then fixed. On the shipped v0.229.0, on demo-hp, an app's Tier-2 copy went from 120 082 104 B (4 database dumps + 3 named-volume tars) to 7 036 B (none of either) in ONE nightly run, and the run recorded itself a success: 'Tier 2 copied docmost -> ... (14.9 KB, 0 leg(s), 0s)'. Evidence: felhom.eu/documentation/audits/DRILL-r403-tier2-delete-2026-08-31/. The mechanism was three individually-correct lines: RunTier2 guards the unit leg with os.Stat only (does the folder exist), rsyncMirror is rsync -a --delete, and nothing between them compared source to destination. An EMPTY unit is a folder that exists. THE GUARD. One predicate, unitCarriesData/unitIsHollow (r403_hollow.go), asking the MANIFEST and never the byte size - a big compose tree with no dumps is dangerous, a tiny unit for a tiny app is fine. Fail closed on an absent or unparseable manifest. RunTier2 skips the unit leg when the source is hollow AND the destination is not; the other legs still run, the run is not failed, and the skip is recorded for the SURFACE (CrossDriveBackup.UnitLegSkipped + UnitPackageDate) as well as logged. --delete STAYS and shrinking stays legal. 07 section 8 row 5's derived-copy rule is unchanged; the fence is exactly one shape. TestR403_DataLegShrinkIsUnaffected is the guard on the guard. THE HONESTY. A preserved package is older than the run that preserved it, so the card carries a notice and the unit-restore confirm names the PACKAGE's date - read from the mirrored manifest's own created_at, not from the status record - plus a clause saying why it is older. THE CAUSE. RestoreTier2Unit now refills a hollow or absent primary unit from the mirror it just restored from, INSIDE the call before returning. The hollow manifest was written two seconds after a restore by the 5-minute capture job; any follow-up job races it. The capture itself is NOT guarded: a capture describing an empty drive as empty is correct, and with the primary refilled there is no hollow state left to describe. Never over a complete primary, never after a failed restore. recordTier2Success and tier2UnitConfirmMsg keep their old signatures as thin callers, so no existing test needed editing. New seam unitRehydrate, separate from tier2Mirror on purpose. 22 new Go tests. Red-proofs run and reverted: A6 (predicate -> size threshold), B1 (guard removed -> the copy's 3 files are DELETED and the seam is called), B6 (a general never-shrink rule -> the shrink case fails), C2 (only-when-hollow dropped -> the complete primary is overwritten).	2026-08-31 14:02:13 +02:00
admin	fed272e62a	docs: REPORT.md — delivery is DONE (golden 0.229.0 vouched, floor raised) ... Section 6 now records the completed delivery instead of an owed bake; observation 2 records that R-242's sixth conviction was paid the same day and the declared --no-verify bypass is historical. New section 15 carries the bake, the round trip, the third independent reader, both proven pre-gates, the three-field vouch, the R-120 gate passing, and the floor proven ACTING.	2026-08-31 12:38:15 +02:00
admin	0957f43d10	docs: record the two documentation commit hashes in REPORT.md section 3	2026-08-31 12:22:02 +02:00
admin	8aa95b5831	docs(v0.229.0): R-102 + R-103 — CHANGELOG, CONTEXT rulings, README, REUSE, REPORT ... CHANGELOG v0.229.0. CONTEXT records three rulings: the source moves and the destination does not; two predicates and not one wider one (R-356's cost restated); and a destructive operation reached from a non-destructive surface carries the difference in the CONFIRM, not the label. README documents the new action and route and corrects the coverage note to the measured count. REUSE maps the four unit-directory-relative primitives and the new manager methods, with the traps. REPORT covers the live drill on demo-hp (docmost, class B, primary unit moved aside — 3 volumes of 3 and 1 database of 1 in 28.65 s, accented filename byte-identical verified as hex, the app reading its own row over TCP; Scenario D with the guest app.yaml also aside, secrets recovered=2/2), the settled count (A=7 B=45 C=1, and why the earlier 9/43/1 was wrong), the five named red-proofs, and seven observations including R-403 and a process error of mine that changed the box and is now in memory.	2026-08-31 12:21:28 +02:00
admin	4c8f0d2919	R-103: the Tier-2 refusal becomes an action ... An app whose Tier-2 copy holds no file legs but a full recovery-unit mirror - 45 of the 53 catalog templates - was told to press a button on a DIFFERENT page. Since R-102 the data it is asking for is restorable from the copy it is looking at. New POST /backup/tier2/unit-restore and backupTier2UnitRestoreHandler: same guards, same restoreOpBlocked() refusal (R-351b), same async shape as the file restore beside it, plus a fail-closed pre-flight so the app is never stopped for a mirror that could not be opened. The outcome reuses unitRestoreOutcomeMsg and adds which copy overwrote the live data. The row offers the action where the refusal was, in a danger style, as a SEPARATE button. The two are not merged: one adds what is missing, the other overwrites. The confirm carries that difference in words and names the copy's date - and says so differently when that date is only an ATTEMPT (R-101). It is built from named Go constants rather than assembled inside an HTML attribute, so a test can assert it verbatim; fmtTimeStr now delegates to a package-level fmtRFC3339Local so the confirm and the outcome cannot render the same date two ways. tier2NoCoverageMsg is NARROWED to the case that remains - no legs and no openable unit - and still names the route that works. tier2UnitNotCoveredMsg is NOT deleted: it is appended where the FILE restore ran and is still exactly true of it. Tests C1-C2 and D1-D6 plus four more. Red-proofs: C1 (widen CanRestore to include HasUnit -> the unit-only cases fail), D6 (drop EndRestoreOp from the handler goroutine -> 'the restore never published a result').	2026-08-31 11:42:03 +02:00
admin	0f9b796615	R-102: the recovery unit on the second drive becomes a way back ... Tier-2 mirrors each app's whole recovery unit to <dest>/backups/secondary/<app>/recovery-unit/ on every run and has done for months. Nothing read it. In the one failure Tier-2 exists for - the primary drive is lost, and the primary unit with it - the surviving copy could not be opened by any action in the product (07-backup-architecture 6.3, 7.2). Part 1.2: RestoreFromRecoveryUnitAt(stack, unitDir) holds the whole body; RestoreFromRecoveryUnit is the thin caller naming the primary unit. ONE implementation, two callers. The SOURCE moves; the DESTINATION does not - live Docker volumes, the live database container, the guest's definition, all unchanged. The R-47 mutation order, the secret reconciliation with unit-over-guest precedence, the fail-closed data-key gate and the no-unit fallback with CountsUnknown are untouched. reimportDBDumpsAtCtx is the bounded-context twin of reimportDBDumpsFrom; the 35-minute bound is now named once so the two paths cannot drift. The R-354 volume-replay seam is reused rather than a second one invented, which is what lets the acceptance test assert the volume leg's source directory. Part 1.3: RestoreTier2Unit resolves the recorded copy, refuses fail-closed unless the mirror carries a parseable manifest - a directory is not a package - and delegates. The single-writer flag is taken inside RestoreFromRecoveryUnitAt, not beside it. Part 2.1: Tier2Coverage gains UnitRestorable and the copy's dates. CanRestore() is NOT widened; it still answers only 'can the file restore run?'. One predicate answering two questions is R-356, which refused 40 running apps for months. Tests: A2-A6 and B1-B5, plus two non-regression guards. The Tier-2 fixtures build their mirror with the production RunTier2, so the claim is 'the copy Tier-2 writes is the copy this restore reads'. Red-proofs: A5 (swap volumes/recreate -> fails on the order), B2 (point the reader back at the primary -> fails with the mirror never reaching the redeploy, and with permission denied once the primary tree is unreadable).	2026-08-31 11:30:34 +02:00
admin	c732006d26	R-102 phase 1.1: split the recovery-unit path helpers, zero behaviour change ... Every unit path helper took (nsRoot, stackName) and joined backups/primary/<stack>/... . That hard-coded 'primary' is the mechanism of R-102: Tier-2 mirrors the whole unit directory to <dest>/backups/secondary/<stack>/recovery-unit/ every night, and because no reader could NAME a unit outside backups/primary/, that mirror has been captured for months and read by nothing. Adds four unit-directory-relative primitives - UnitComposeDir, UnitManifestFile, UnitDBDumpDir, UnitVolumeDumpDir - each taking the recovery-unit DIRECTORY itself. The four existing (nsRoot, stackName) helpers become thin wrappers over them and keep their exact signatures and their exact return values; every current caller compiles untouched. ONE implementation, two callers - the rule restoreDockerVolumesFrom already follows in this repo. TestR102_PathWrappersAreByteIdenticalToToday pins the wrappers against hand-written literals (not re-derived from the helpers under test). Red-proof: UnitComposeDir join changed to 'compose2' -> the test fails on all three fixtures.	2026-08-31 11:16:02 +02:00
admin	430fb4448d	REPORT.md — golden 0.228.0 baked, vouched, floor raised; demo-felhom self-updated	2026-08-31 10:54:19 +02:00
admin	04abeb20c0	REPORT.md — correct observation 4: the memory was right, I did not follow it	2026-08-31 10:39:36 +02:00
admin	0989bb83c3	REPORT.md — fill in the felhom.eu commit hash	2026-08-31 10:39:13 +02:00
admin	b670da748d	REPORT.md — v0.228.0 (R-399 + R-400): proven live on demo-hp at both depths	2026-08-31 10:38:49 +02:00
admin	3c49dc8ea4	v0.228.0 — the off-site check reads the data; the debug page stops lying (R-399 + R-400) ... R-399: monitoring.integrity.read_data_subset defaults to 100%. A pack damaged without changing its size made plain `restic check` report "no errors were found" on demo-hp 2026-08-30; every read-data form caught it. Cost on that 134 MB store: 35.0s structure vs 39.2s at 100%. "off" (any case) is the off token; empty means not-configured, therefore the default; a malformed value falls back to the DEFAULT, never to structure. A completed check over 5 minutes logs a WARN naming the duration, the depth and R-401 — operator log only, no hub event, no depth change. The depth is now recorded with the verdict (LastIntegrityDepth; empty = NOT RECORDED, never "structure"). R-400: 24 debug-page references, 17 dispatched, 7 dead — three of which fetched on page LOAD, so those panels were permanently blank. backup/crossdrive implemented; backup/infra, hub/infra-push, dr/infra-status, storage/watchdog-status and both storage/simulate-* deleted with their panels and JavaScript. scripts/debug_route_gate.py fails in both directions and is registered after the seven were resolved. 18 referenced, 18 dispatched, none orphaned. Corrections: the dead-field warning in report/types.go said the controller runs no integrity check and the notifiers are called from nowhere — both false since v0.227.0. controller.yaml.example gains its missing integrity: block. integrityCheckTimeout's "ships OFF" comment rewritten.	2026-08-31 10:24:29 +02:00
admin	300d7e87d7	REPORT: R-359/R-397 shipped and validated; the measurement changed what it is worth ... The headline is not the feature, it is what measuring it revealed: THE STRUCTURE CHECK THAT SHIPS ON DOES NOT CATCH SILENT CORRUPTION. A pack corrupted without a size change returned `no errors were found`, exit 0. Only --read-data caught it. So R-399 is not merely a bandwidth question -- at the shipped default a class of damage is not checked at all. The three numbers R-399 needed are MEASURED, not estimated: store 134.3 MB / 67 snapshots; structure check 35.0 s; and the full curve 10% 35.9 s, 50% 37.3 s, 100% 39.2 s. At this size re-reading everything costs four seconds more than reading none. Stated limit: they do not extrapolate. Records four things that went wrong and were caught rather than shipped: - R-398 was MY OWN mistaken row. The seam already existed, Part 0 was not built, and building it would have HIDDEN the unlock --remove-all escalation from the assertions that must see it. Corrected, not closed. - the damage classifier matched restic's ORDINARY progress output; the negative control caught it (v0.227.1). - an exit code I misread through a pipe, corrected by re-measuring. - wire-contract convicted a field the hub cannot decode; allowlisted WITH A REASON rather than skipped, because building the hub display is a decision R-331 already ruled belongs to the operator. And the sweep the task asked for: EIGHT debug buttons post to endpoints that do not exist, not one. 24 referenced, 17 dispatched. Filed as R-400. Not live-validated and each says why: the weekly firing (a week away), read-data on a large store, and the join between "restic catches it" and "my code classifies it" -- both proven, the join is not, and the seam is named.	2026-08-30 21:29:15 +02:00
admin	45770f2282	v0.227.1: the damage classifier matched restic's ordinary progress output ... A patch and not a rebuilt 0.227.0: that tag was already running on demo-hp, and re-pushing changed bytes under a live tag is the :latest hazard with extra steps. looksLikeRepositoryDamage matched bare "pack ", "tree ", "snapshot ", "blob ". A HEALTHY restic check prints "check all packs" and "check snapshots, trees and blobs" -- so any check that failed for a NON-damage reason, a connection dropped mid-run for instance, would have been classified as a corrupted repository and told the customer their backups may be damaged. That is the false alarm that teaches an operator to ignore the true one. Caught by the NEGATIVE control, built from the real bytes of a real passing check on demo-hp. The spec made the negative control mandatory and this is what it was for: a control that has only ever seen the failing case proves nothing. Signatures are now phrases from restic's own error wording. Also in this commit: CONTEXT.md records the three rulings (take the flag and skip, due-ness not a weekday, publish on OffboxReportStatus not the R-331 dead fields) plus the measurement a future session would otherwise assume wrongly -- THE STRUCTURE CHECK DOES NOT CATCH SILENT CORRUPTION. README documents the job, the route and the config, and corrects a line that listed four debug backup routes when only two exist. REUSE gains three rows, including one that records R-398 was my own mistake so nobody re-files it.	2026-08-30 21:22:23 +02:00
admin	0d52a42c17	R-359 + R-397: the off-site store gets checked, and the advertised check becomes real ... Nothing ever verified that the off-site copies are still readable. The whole-guest tier has verify jobs; the tier holding the customer's documents and photos had none -- the complete set of restic verbs this controller used contained no `check`. We would have found out at restore time, with a customer waiting. On 2026-08-21 a deliberately damaged pack was caught at once by plain `restic check`; we had never run it. R-397: NotifyIntegrityOK/NotifyIntegrityFailed existed with no caller, the hub allowlists both event types and carries the Hungarian text for both, the settings checkbox exists, and the debug button posts to /api/debug/backup/ integrity. Everything was built except the part that runs. SIXTH instance of that shape in this project. THE HAZARD SHAPES THE WHOLE DESIGN. resticStep self-heals a crash lock by running `unlock --remove-all` and retrying, and its own comment records why that is safe: every caller holds the in-process single-flight mutex, so any lock it meets is stale. A check that did not take that flag could meet a LIVE prune's lock from this same box, remove it, and retry over the top of it. So the check TAKES THE FLAG and SKIPS rather than waits -- waiting would pin the nightly backup behind it, and a skip costs nothing because due-ness makes tomorrow try again. TestR359_SkipsWhenRunningFlagHeld asserts the NON-EFFECTS: restic never invoked, `unlock` never in any argv. Its red-proof prints the real thing -- restic running `check` while the flag was held. DUE-NESS, NOT A WEEKDAY. Daily job, weekly behaviour: "is the last successful check older than 7 days?" not "is it Sunday?". R-341 is exactly the other shape, a dated check quietly missed and never caught up. No Weekly primitive added. THREE OUTCOMES, NOT TWO. Skipped, Unreachable and failed are different facts. "I could not look" is not "I looked and it is broken" -- R-339 already owns reachability, and a second alarm for the same fact trains the operator to discount the one alarm that means the backups are damaged. A timeout is unreachable, never damage. A failure advances due-ness (a broken store must not be re-checked nightly); a skip and an unreachable store do not. Success is severity `info`, which severityNotifies DROPS -- it mails NOBODY, by design. A weekly success e-mail is how people stop reading their alerts. The customer gets a SENTENCE; restic's words go to the log, truncated (R-379: 615 bytes of raw database text reached a customer once). read-data-subset ships OFF and a malformed value is refused at read time rather than handed to restic, where one typo would fail the whole check. Published on OffboxReportStatus, NOT on report.BackupReport's IntegrityOK -- those were retired by R-331 YESTERDAY and TestBackupReport_DeadFieldsStayZero still passes unmodified. Also: the monitoring page stopped promising a Sunday job that never existed, and the debug button got its dispatch case. PART 0 WAS NOT BUILT, AND R-398 WAS MY OWN MISTAKE. The seam it asked for already exists: offboxRunner/SetOffboxRunner/m.runner() has been injectable since the off-site tier shipped, and other tests drive restic-backed paths through it. A resticStepFn seam would have been WORSE here -- it would replace the `unlock --remove-all` escalation and hide it from the assertions that must see it. R-358's AST ordering test is converted to a real execution test instead, which immediately surfaced something the AST walk could not: unlockStale legitimately runs before the restore. Four red-proofs, each printing the pre-fix behaviour. Green gate: 28 packages, rc 0. All 12 controller gates OK.	2026-08-30 21:03:29 +02:00
admin	e64c84aef8	REPORT: the golden debt named in section 12 was paid the same day ... Golden 0.226.1 baked, published, round-trip verified, vouched, floor raised. golden_currency_gate.py red -> green on the same command, so the three declared --no-verify bypasses are historical rather than standing. The fleet state changed with it: both demo machines run 0.226.1, and demo-felhom got there by SELF-UPDATE rather than by hand -- which is the positive observable that the floor is acting and not merely set.	2026-08-30 20:24:14 +02:00
admin	c476ae51a5	REPORT: record the 0.226.1 patch and which version the live evidence describes ... The section 6 live validation ran against 0.226.0, which was already on the box when the fallback defect was found. Saying so rather than re-attributing the evidence to 0.226.1: they differ only by CountsUnknown and its two tests, and neither touches any path that validation exercised.	2026-08-30 19:53:21 +02:00
admin	36c13cfa0a	v0.226.1: ship the CountsUnknown fix under a NEW tag, not a rebuilt 0.226.0 ... 0.226.0 was already deployed to demo-hp by hand when the fallback defect was found. Re-pushing a changed image under a tag that is already running somewhere is the :latest hazard with extra steps -- two different images, one name, and no way for a box to tell which it has. So the fix ships as a patch.	2026-08-30 19:51:52 +02:00
admin	c0c8fe67bf	An unknown drawn as a zero: the defect v0.226.0's own fix introduced ... Writing the REPORT's observation "the no-unit fallback already reports a zero result, which is honest" exposed that the sentence was FALSE. A zero UnitRestoreResult is Scenario B's shape. So RestoreFromRecoveryUnit's fallback to RestoreApp -- which returns only an error, and whose signature is deliberately out of scope -- would have printed "ez a mentes csak a beallitasokat tartalmazta, adatot nem" over a restore that may have replayed the app's entire dataset. That is an unknown drawn as a zero: the exact R-88 failure direction this whole change exists to remove, re-introduced by the change. UnitRestoreResult now carries CountsUnknown, the fallback sets it, and there is a fourth sentence claiming only what is known -- the restore ran, the app is back, and we cannot say what came back. RestoreApp's signature is untouched. Pinned by TestUnitRestoreOutcome_NoUnitFallbackSaysUnknownNotEmpty. The A5 seam test was corrected too: its fixture has no recovery unit, so it exercises exactly this path and had been asserting the wrong sentence -- it now asserts the unknown, which is what pins the fallback to it. IT WAS THE observations GATE REFUSING THE PUSH THAT FORCED THE RE-READ. A gate written to stop findings dying in an overwritten REPORT.md caught a live defect instead. Also files R-397 (NotifyIntegrityOK/Failed are dead code AND the monitoring page advertises a weekly integrity check that does not exist) and R-398 (resticStep is not a seam, which is why R-358's ordering needed an AST test) rather than leaving them in a file that is overwritten every session. REPORT.md is the full run record: baselines re-confirmed, per-test results, the five red-proofs with their observed output, the live validation with verbatim Hungarian messages, what was NOT validated and why, teardown across three layers, and the register 165 -> 167 -> 161. Green gate clean: 28 packages, rc 0. All 12 controller gates OK.	2026-08-30 19:51:16 +02:00
admin	e4e0aa8f46	REPORT: v0.226.0 shipped, three of four fixes proven live on demo-hp ... Records what was validated and, in equal detail, what was not. PROVEN LIVE (demo-hp, endpoints the UI invokes, evidence copied off the box): R-353 "A(z) opengist: 1 adatkotet visszaallitva -- az alkalmazas ujraindult." read off the customer's own wizard page, with real counts 1/1 volumes and 0/0 databases and correctly no database clause. R-360 refused in the exact flag state that produced the bug, and the planted canary file survived -- the consequence, not the branch. R-358 a mode=unit restore wrote {"schema":1,...,"full":false} at mode 0600 with no .tmp left, and the gate logged place-to-live closed. NOT live-validated, and each says why rather than being omitted: R-357 filling a real filesystem is a drill step, not a build step. R-353 Scenario B NO app on demo-hp still has a data-less unit -- the spec named opengist from 21 August and it has since been recaptured (now 1 volume dump). Manufacturing one means falsifying a manifest, which is the hand-set-state shortcut this project forbids. R-353 Scenario C and R-358's failed-download branch: unit-tested only. Also recorded, because a near-miss that is quietly fixed teaches nobody: the first B1 red-proof exposed a HOLLOW TEST OF MY OWN. With the gate removed the run refused earlier, at the placement stat pre-pass, so `stops == 0` passed against the pre-fix code. Fixture corrected and assertions reordered; only then does the red-proof print THE APP WAS STOPPED (1 call(s)). Register 165 -> 167 -> 161. Six rows compressed into CLOSED-ITEMS keeping title, version, evidence and every sentence stating a rule; full original at `git show e027b5d9`. No open row touched. ROADMAP not edited -- none of these four ever had a row there, stated rather than silently skipped. Teardown: this run provisioned nothing, across all three layers. Two throwaway scripts and one canary directory were planted in guest 9201 and both removed.	2026-08-30 19:41:48 +02:00
admin	b8af72764d	R-353/R-357/R-358/R-360: the restore tells the truth (v0.226.0) ... Four defects on the restore surface, all proven on demo-hp during the 2026-08-21 backup-truth drill, all still in shipped code. They share one acceptance idea: a restore surface must state what it actually did, and must refuse what it cannot do. VERSION NOTE. The task specifying this targeted v0.224.0 against baseline f8c9390. Both were consumed earlier the same day by R-330 (0.224.0) and R-331 (0.225.0). Drift re-confirmed against live Gitea before the first edit, operator authorised proceeding, every symbol the spec named re-verified present at the real baseline e5eee50. R-353 -- a restore that gave back nothing still said it worked. RestoreFromRecoveryUnit returned only error, so the surface printed "<app> visszaallitva (<snapshot>)." -- equally true of a run that returned an entire dataset and one that returned nothing. The count already existed and was discarded one line deep: restoreDockerVolumesFrom always returned it, the wrapper threw it away. Now (UnitRestoreResult, error), carrying replayed counts AND what the manifest LISTED, because zero-replayed has two causes that are opposite news. Three cases, three sentences, and EVERY one is a claim about the BACKUP, never about the app -- this path has no SafetyDump discriminator, and 07-backup-architecture 6.3 records that an absent dump says nothing about the app (R-361 destroyed canonical .sql files for four months). R-357 -- the destructive restore had no free-space gate. offbox_reconstitute.go contained ZERO references to offboxFree; all three existing gates guard non-destructive paths. The gate now sits before mapOffsiteRestorePaths, writeSafetyDump and StopStack, so a refusal costs nothing. Position IS the fix, which is why the test asserts StopStack was never called. No headroom multiplier (matches PlaceOffsiteRestore; the x1.1 elsewhere predicts a download). Fail closed on either probe <= 0 -- otherwise `free < need` with need==0 is FALSE and an unmeasurable scratch sails through: a gate present and inert. R-358 -- a failed download was offered as a good one. The gate answered "the directory exists and is non-empty", which is exactly what a part-way restic run leaves. Now a completion marker written 0600 atomically AFTER restic returns nil, with any stale one cleared BEFORE it starts; both orders pinned by an AST test because resticStep is not a seam. Both handlers refuse server-side: the wizard flags control a button, and a hidden button is not a guard. SCENARIO F ANSWERED, and worse than the question assumed: a unit-only scratch IS reachable through the real flow, by the most ordinary route. "Ellenorzo visszaallitas" (mode=unit, advertised non-destructive) writes the SAME directory -- offboxRestoreScratchDir ignores `full` and --include limits what restic extracts, never where -- so a customer who ran the SAFE restore was then offered the destructive one over a unit-only copy. Filed R-396; the marker closes it. R-360 -- the delete refused only while a BACKUP ran. IsRunning() is FALSE for the whole of a verification restore; the five sibling handlers all use restoreOpBlocked(). Its doc comment claimed it already did this, which is why nobody looked -- corrected in place. Red-proofs, each printing the pre-fix behaviour, in CHANGELOG and REPORT. The first R-357 red-proof exposed a hollow test OF MY OWN and it is recorded rather than quietly fixed: the fixture refused earlier at the placement stat pre-pass, so `stops == 0` passed against the pre-fix code. Fixture corrected, assertions reordered so a removed gate reports the outage rather than "no error returned". Green gate clean: 28 packages, rc 0. All 12 controller gates OK.	2026-08-30 19:31:31 +02:00
admin	e5eee501b5	R-331 (controller half): forward stats_known so the hub can tell empty from unmeasured (v0.225.0) ... The hub's operator Backup card read `Snapshots 0 / Repo Size 0 MB / Integrity Unknown` for EVERY customer, because it rendered the report's `backup` object -- whose snapshot/size/integrity fields have had NO producer since disk-tier restic moved to the host agent (slice 8C). buildBackupReport leaves them zero deliberately and says so. Measured on demo-hp 2026-08-30 while that night's log said `[offbox] backup OK: 8 app(s) backed up, 67 snapshot(s), 2m14s`. The live numbers were always in the report's `offsite` object, which the hub already reads for its Offsite page and its fill/staleness alarms. The hub fix is to render that -- and that made exactly ONE field mandatory that was not being forwarded. snapshot_count:0 means two opposite things: "holds nothing" and "never measured". R-225 measured that confusion inside this repo (a rebuilt box rendered 0 pillanatkep over a store really holding snapshot f3d9cd67), and settings.OffboxTarget.StatsKnown fixed it for the controller's own UI. It was never put on the wire, so the hub was free to make the identical mistake one layer up -- and did. OffboxReportStatus.StatsKnown now carries it, omitempty, so an older controller sends no key and a reader degrades to UNKNOWN, never to EMPTY. Absence is ignorance, not emptiness. The four dead BackupReport fields stay on the wire (historical reports in the hub store must keep parsing) but now carry a warning naming R-331 and pointing at Offsite. TestBackupReport_DeadFieldsStayZero fails the moment a producer appears for one -- the prompt to update the hub card in the SAME change rather than ship a field nothing renders. RED-PROOF: drop `StatsKnown: t.StatsKnown` -> "a MEASURED empty repository reported stats_known=<nil>". Tests assert the JSON the hub sees, not the Go struct: measured-empty and never-measured must differ ON THE WIRE, which is the entire point of the field. Green gate clean: 28 packages, rc 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LB8FmJaGd2cyjvy6dbEjpM	2026-08-30 18:38:10 +02:00
admin	45b52b6ed5	R-330: live validation on demo-hp — 3 scans inside the window, 0 events ... v0.224.0 deployed to both demo boxes (both `0.224.0 … (healthy)`). Proven through POST /api/backup/run, the endpoint the UI button invokes: 8 stacks stopped and restarted over 87s, three dead-app scans ran INSIDE that window (16:09:26 docmost, 16:09:56 paperless-ngx, 16:10:26 romm -- the same three apps that alarmed the night before on 0.223.0), zero app_start_failed pushed. The scan count is the positive control, not decoration: an absent alarm is equally consistent with "suppressed correctly" and "the scanner stopped". A first run is discarded IN THE REPORT rather than quietly dropped -- it fired 52s after a controller restart, inside deadAppBootGrace (90s), where the scan returns early and could not have alarmed whatever the code did. demo-felhom is deployed but NOT independently proven and says so: its single app cycles in ~1s, too fast for any 30s scan to land inside. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LB8FmJaGd2cyjvy6dbEjpM	2026-08-30 18:12:26 +02:00
admin	92cebb8c95	R-330: stop the backup alarming about the apps it is holding down (v0.224.0) ... Measured live on demo-hp 2026-08-30 (controller 0.223.0): the nightly db-dump and offbox-backup legs stop each stack ~13s to tar its volumes while the deadapp-check job scans every 30s, so the scan caught whichever stack was mid-cycle and pushed app_start_failed to the customer. 61 e-mails about apps that were never broken. The defect is not a missing mechanism. quiesce/suppress.go solved exactly this in v0.179.0 and works -- but classifyRunStates read only the quiesce loop's set, and that loop covers the WHOLE-GUEST backup. The per-app legs stop stacks through Manager.DumpAppVolumesSafe, which registered with nothing. Two mechanisms stop apps on purpose; only one told the alarm. Fifth instance of the "seam built but never wired" class, and the first where the unwired half was a consumer. The suppression now rides AppStopGuard, which already brackets every deliberate stop in the product (Begin before the stop, End after a successful restart) at all three call sites, and which main.go hands as ONE object to the backup manager and the exporter. scanDeployedAppRunStates takes the union of both sets. All three per-app stop paths are covered, not only the reported nightly one. It cannot latch -- End() runs only on a restart that SUCCEEDED, so unlike the quiesce loop an open-ended hold is a real hazard here: 1. ReleaseFailed drops the entry IMMEDIATELY on a restart that broke, wired at every failure path, so the app alarms on the next scan; 2. Begin REPLACES the set (one marker file = one operation); 3. appStopMaxHold (6h) caps a hold nothing released, logged at WARN. Grace is 180s, deliberately quiesce's own constant and derivation. Suppression is NOT persisted: after a crash the guard holds nothing and a down app must alarm. ReleaseFailed keeps the durable crash marker; a test pins that. Three companion red-proofs, each printing the pre-fix value (REPORT.md section 5): - drop markStopped from Begin -> "suppressed at stop = map[]" - drop ReleaseFailed from the dump -> "map[bookstack:true] after a restart that FAILED" - pass nil instead of appStopGuard -> the AST wiring test fails The third is load-bearing: the component was never the broken part, so a suite that only injected it would have been green against the shipped defect. Green gate clean: go build + go vet + go test ./... -- 28 packages, rc 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LB8FmJaGd2cyjvy6dbEjpM	2026-08-30 17:58:15 +02:00
admin	f8c9390946	gate 11: register the observations gate, and mark up this repo's observations ... The shared gate lives in felhom.eu/scripts/observations_gate.py and is invoked across the workspace, exactly as reuse_refs_check.py and instructions_gate.py already are. It is never copied. REPORT.md's observations now carry their markers. Item 1 was the finding that had no register row - only the first broken app per hour reaches the operator - and it is now R-389. Item 4, the golden-bake runbook's missing `pveam update`, is R-390. Items 3 and 5 are declared NOT-A-FINDING with their reasons. The observations' text itself is unchanged; only the markers were added.	2026-08-23 13:53:18 +02:00
admin	1da2c9c6c6	docs(v0.223.0): REPORT, CONTEXT rulings, README severity contract ... REPORT overwritten: the 1.1 sweep in full (one bad severity, nine legitimate "warn" strings that are healthcheck statuses), the hub manifest's real location since the task's premise was wrong, all five red-proofs with the layer each guard sits at, the live walk in six steps with the hub's own records quoted, and the absent-intent count (0 of 8). Three things are reported that a tidier account would omit: red-proof 5 passed first time because the mutation was INERT; Scenario G was silently refused twice behind an HTTP 200; and the live Scenario A does NOT prove the customer gate, because demo-hp has no prefs row at all. CONTEXT records the severity vocabulary as a ruling with its mechanism, the intent ruling with its three-way handling of unknown, both fences, and two traps worth more than the fixes: a 200 can be a refusal, and a passing red-proof can mean an inert mutation. README: the event table said `app_start_failed | warn` - the defect, written down as if correct. Now `warning`, with the vocabulary contract and who receives what. `disk_critical` also corrected from `error` to `critical`, which is what fillwatch has always sent.	2026-08-23 12:06:45 +02:00
admin	9832760027	v0.223.0: the app-down alarm reached nobody (R-329), and the stop nobody heard (R-386) ... R-329. NotifyAppStartFailures emitted severity "warn". The hub accepts exactly {info, warning, error, critical} and silently coerces anything else to "info", which severityNotifies then drops BEFORE both legs. Banner shown, event stored, POST 200, no mail sent. One word. This is the second time: DiskAlertKind.Severity emitted "warn" until v0.215.0 and its own comment records that every warning-level disk alert went to nobody. A comment recorded the lesson and nothing enforced it. The guard is now an AST walk over the whole controller - grep cannot work here, since "warn" appears legitimately nine times as a healthcheck status vocabulary. The sweep found exactly one bad severity. Its limits are stated: the walk cannot follow a variable, so all six dynamic call sites are registered by name with the values each can take, and a new one fails the test. Two of the six were found by the guard, not by the hand sweep before it. Also pinned: fillwatch.Band.Severity() returns "" for BandOK, which would vanish the same way. It is unreachable because Check() notifies only on escalation - but that safety lives in a different function from the one that looks unsafe, so the test asserts the consequence rather than the mapping. app_start_failed gains a customer toggle, DEFAULT OFF, per operator ruling. The operator is mailed either way: processOperator never consults customer prefs. It is deliberately NOT in operatorOnlyEvents, which would make the toggle a lie. R-386. classifyRunStates decided "the customer stopped this" from the STATE, so every stopped stack was assumed deliberate. Measured on demo-hp: privatebin stopped out of band, nine scans, zero events, zero banner - while the comment beside it claimed an out-of-band stop still alerts. DesiredState already records the answer and has exactly one writer. Stopped -> no alarm; Running -> alarm; absent -> UNKNOWN, keep today's behaviour AND say so. Absent stays silent deliberately: reading it as "nobody asked" would email about every app anyone ever stopped, fleet-wide, on the first cycle after upgrade. The gap is bounded not silent - IntentUnknown is set and the names are logged at INFO on the heartbeat cadence. failedRestart still lifts a Stopped intent, or F-CRIT-1 re-opens. No new DesiredState writer. Two settings toggles each governed two alarms. "Lemez figyelmeztetes (90%+)" also wrote disk_critical, the drive-is-FAILING alarm. Now four honest toggles; 12 became 15. A no-op save stores the existing slice verbatim, so byte identity is by construction - without that guard the defaults case reorders, which the red-proof caught. Test count 1504 -> 1522. Five red-proofs, five seen failing; one passed first time and is reported - that mutation was inert, not the test weak.	2026-08-23 11:21:06 +02:00
admin	14137efac5	docs(v0.222.0): REPORT, CONTEXT decisions, README state table + the ordering ... REPORT.md overwritten with the full run: baselines and the hub's four numbers, the four red-proofs with the mutation and observed text for each, the five IsDownState consumers walked and named, the live walk in full with the old and new heartbeat lines quoted side by side, and the halt. CONTEXT records the decision - a dead supervised member is asked about before a failing healthcheck, because they are different questions and the second was answering the first - plus the fence that IsDownState did not move, the trap that three existing subtests pinned the defect, and R-386. README gains the `degraded` row, which the state table never had, and a note that the ORDER is load-bearing. Points at the new alarm-ladder architecture doc.	2026-08-23 07:58:09 +02:00
admin	5da11c4480	v0.222.0: ask whether a supervised member is DEAD before whether one is UNHEALTHY (R-384), and stop promising an undo copy nobody looked for (R-383) ... R-384. aggregateState returned StateUnhealthy the moment unhealthy > 0, and the R-51 mixed-case block that asks "is a supervised member dead?" sat below it. A two-container app whose database exits goes unhealthy BECAUSE it cannot reach that database - so the symptom the dead database causes was what suppressed the alarm for it. unhealthy is not a down state, so classifyRunStates never marked the app down and app_start_failed never fired. Measured live on demo-hp 2026-08-22: bookstack-db stopped at 21:27:01 and the F-OBS heartbeat printed "0 currently down" throughout. R-51's 18-hour immich failure, back through a different door. Two things moved, and either alone leaves the defect standing: the supervised test is hoisted above the unhealthy/starting/restarting returns, and "some members are up" now counts ANY member not in the down bucket. The old guard was running > 0, which made the R-51 block unreachable in exactly the case it was written for. IsDownState is byte-identical - unhealthy stays excluded, because an unhealthy container is running and folding it in reintroduces the flapping that exclusion exists to stop. No new state was minted. Only the ORDER changed. The priority comment was rewritten because it asserted an ordering the code no longer has. Three subtests in TestAggregateState_UnchangedBranches were AMENDED: they asserted an unhealthy/starting/restarting member beat an exited peer on unless-stopped, which pinned the defect as settled behaviour. They keep their intent with the down member given a benign policy. R-383. The double-failure message said the previous state's backup EXISTS, built from the returned path without asking the filesystem - and a missing file is one of the two ways that rollback fails. undoCopyPhrase now describes the copy from disk: present, partial, missing (still naming where it should be), or never written. Zero-length counts as missing. Test count 1494 -> 1504. Four red-proofs planted, four seen failing; the two halves of R-384 convict independently.	2026-08-23 07:25:00 +02:00
admin	da75603553	R-385 record: v0.221.1 gets its own CHANGELOG heading ... 0.221.1 was built, baked and vouched on 2026-08-23 with no entry of its own. The prune-ordering fix (commit 810b18a) was written inside the v0.221.0 entry, so the newest heading in the record named 0.221.0 while the fleet ran 0.221.1. The image was never in question - the record was. The reasoning is MOVED, not rewritten: the paragraph is verbatim from the v0.221.0 entry, which no longer claims it. The new entry also names the pinning test and its red-proof. Pushed alone and before anything else this session, so the record is correct even if the rest of the session does not land.	2026-08-23 07:06:18 +02:00
admin	f7881787f4	R-361 docs: CONTEXT decision, README, REPORT ... Records the db_dumps decision with every consumer named, the trap that a stable db_dumps lets CaptureRecoveryUnit's already-current early return fire (so per-capture housekeeping must sit above it), and the NEGATIVE that a held app does not raise the dead-app alarm - measured, not reasoned, so nobody re-derives it.	2026-08-23 00:26:32 +02:00
admin	810b18ab8e	R-361 follow-on: the undo-copy prune must run above the already-current check ... Excluding pre-restore-* from db_dumps made that list stable across restores, so CaptureRecoveryUnit's already-current early return began firing where it never had - and the prune, which sat after it, stopped running in exactly the case it exists for. Measured on demo-hp minutes after the change: four undo copies on disk against a cap of three. The prune is housekeeping on the dump directory and is independent of whether the manifest needs rewriting, so it belongs above the check. Pruning cannot disturb dbDumps, which no longer contains those names. Pinned by TestR361_UndoCapHoldsWhenTheUnitIsAlreadyCurrent; its red-proof moves the call back below the return and the cap fails at 5.	2026-08-23 00:02:32 +02:00
admin	968c968559	R-361: the safety dump destroyed the app's own database backup ... writeSafetyDump called DumpOne into the app's OWN unit dir and renamed the result to pre-restore-* afterwards. DumpOne writes <stack>-<dbtype>.sql - the app's canonical dump - so every safety dump overwrote the app's real backup and then moved it away, leaving the app with no database backup until the next nightly run. A local restore-from-unit in that window tells the customer the app never had a database. The comment beside it asserted the rename meant it 'can never overwrite the app's real dump'. False as written, and believed for four months. Measured live before the fix: docmost and bookstack each held only pre-restore-* files and no canonical dump. DumpOneTo takes the final path and derives its own .tmp from it. DumpOne keeps its signature and calls it with the canonical name. writeSafetyDump asks for its own name directly; the rename is gone; the comment now states the invariant and how it is enforced. db_dumps no longer lists the undo copies. All three consumers of Manifest.DBDumps were grepped and named - all inside recovery_unit.go, none reads it for recovery. The files are neither deleted nor hidden. Tests 1485 -> 1493. FIVE red-proofs, TWO PASSED first time and both are reported: the behavioural tests inject the dump seam so a mutation inside DumpOneTo was invisible, and 1.3 had no test at all. Guards added at the layer each defect lives in; both mutations then convicted.	2026-08-22 23:39:59 +02:00
admin	2024ed9982	CONTEXT + README: the failure ladder and the operator ruling (v0.220.x)	2026-08-22 18:40:33 +02:00
admin	1a2405e86f	R-379: --clear-restore-hold now states the required restart ... It runs as a second process: it clears settings.json but the running controller keeps its in-memory copy and goes on refusing. Measured on demo-hp - clear succeeded, file correct, start button still refused until a restart. Also records the lost-update window between the two processes, and why clearing through the running controller (the right shape) needs an operator tier the controller's HTTP surface does not have.	2026-08-22 18:21:47 +02:00
admin	5b52a5964d	R-379 fix: the rollback must re-discover the DB container ... Found by v0.220.0's own live walk on its first real run. writeSafetyDump captures its DiscoveredDB before the stop; the DB-only start then re-creates the container with a new id, so the rollback's docker exec hit a dead container and sat in waitDBReady for 30s. The app was held for an infrastructure reason while its data was recoverable. Re-discover and match by {stack, engine} - what reimportDBDumpsFrom already did. Fail closed when the container cannot be found. No unit test caught it because they all inject the import seam and never look at container identity. The new test asserts the identity handed to the import.	2026-08-22 18:14:31 +02:00
admin	2c724c9283	R-379/R-380: put the customer's undo copy back when a database restore fails ... R-379 and R-380 were one failure. Both ended with a half-restored database; the only difference was whether it looked broken. Postgres emptied and crash-looped; MariaDB applied part of the dump and reported health=healthy with a zero-row schema-version table. Measured live on demo-hp 2026-08-22. The undo copy was already taken and already good - proven by hand that day on both engines. Nothing in the product could apply it. Now it does, with the same ImportDump call, before any restart and inside the DB-only window. The WHOLE undo set, matched on this run's stamp. writeSafetyDump returned one path for an app with two databases; a rollback on that would restore one and leave the other half-written. When the rollback also fails the app is HELD STOPPED (operator ruling): a running app on a half-written database lets the customer make the damage permanent. Every start path refuses it - customer button, appstop Recover, boot sweep - via the shared driveStartGate, checked ABOVE its driveless early return because these apps have no drive. The marker is ended so nothing auto-restarts it. The row goes red. Cleared with --clear-restore-hold, an operator CLI route. --single-transaction is a belt on Postgres only; MariaDB DDL is not transactional and that is why the rollback is the fix. R-381: the engine's stderr stops reaching the customer (615 bytes on MariaDB, its middle rows out of their own database) and starts reaching the operator log, which never had it. R-382: the summary log prints the volume count it already held. Undo copies resolve to their own app, are marked IsUndo, and are capped at 3 per app, pruned from the capture side. The reported render-as-an-app symptom did NOT reproduce - the live page was read first and had zero occurrences. Tests 1468 -> 1483. Eight red-proofs; ONE PASSED and is reported: the R-381 behavioural test injected below ImportDump. A guard at that layer now convicts.	2026-08-22 18:03:18 +02:00
admin	0f3cf0dbb2	CONTEXT: R-356 design decision (v0.219.0)	2026-08-22 13:26:57 +02:00
admin	c1dbb05ad6	README: where an off-site restore puts the data (R-356, v0.219.0)	2026-08-22 13:26:44 +02:00
admin	cbc3fa589c	docs: R-356 CONTEXT + REPORT (v0.219.0, proven live on demo-hp)	2026-08-22 13:25:36 +02:00
admin	08eb1a6e3a	R-356: the off-site restore refused every app that has no data drive ... ReconstituteFromOffsite and PlaceOffsiteRestore both resolved the restore destination with the RAW HDD_PATH and read an empty answer as "the app is not installed". For 40 of the 53 catalogue apps that answer is correctly empty and permanent, so both actions refused forever for a running, healthy app — and told the customer to reinstall it "in the same place", which those apps never offer. Separate the two questions. "Installed?" is asked of ListDeployedStacks via a new Manager.isStackDeployed that fails CLOSED on a nil provider. "Where?" is answered by GetAppDrivePath — the same resolver CaptureRecoveryUnit wrote the snapshot with, so the restore aims at the place the backup came from. The 13 drive apps are unchanged: own drive, mismatch check, ack still required. A third refusal, with its own sentence, covers installed-but-no-resolvable-root. Fixtures that marked an app "installed" by giving it an HDD path now state deployment as its own fact. No assertion weakened.	2026-08-22 13:08:46 +02:00
admin	2da259af38	docs(v0.218.0): README names the volume leg and the compose-project attribution; CONTEXT records the blocker ... The README's reconstitution sequence gains the volume replay it never had, and the AppBackup row states that DiscoverDatabases now prefers the compose project label. CONTEXT records the thing an operator most needs next: R-354's fix cannot reach the 40 apps that need it most until R-356 is closed, because the off-site restore still refuses outright for every app that declares no data drive. The live confirmation was therefore done on calibre-web and paperless-ngx.	2026-08-22 09:58:57 +02:00
admin	5ce3a44645	v0.218.0: attribute a DB container by its compose project, and replay volumes on the off-site restore ... R-355 (first, because it is the only one where data can be lost for good). paperless-ngx's PostgreSQL was dumped into backups/primary/paperless/db-dumps/ — a directory for a stack that does not exist, on the system drive — while the app's own unit recorded db_dumps: null. The same misattribution reached writeSafetyDump, so a destructive restore of that app took NO undo copy and the fail-closed refusal was never reached. Fixed by reading the compose project label, which is the stack name by construction (compose runs with cmd.Dir set to the stack dir and no -p). The old derivation stays as the fallback and an unresolvable attribution is now loud. Catalogue sweep, proven able to convict: one affected app of 53. The fix is in the controller, not the catalogue. R-354. ReconstituteFromOffsite skipped every unit placement and the volume archives live inside the unit, so the off-site restore had no volume leg at all — proven live with planted files: calibre-web's 1,422,848-byte config archive was in the unit, the snapshot and the checking folder, and the restore reported success without it. For the 40 of 53 apps that declare no data drive that archive is the whole dataset. restoreDockerVolumesFrom is the local path's own replay with an explicit directory: ONE implementation, two callers. Volumes replay before the database and inside the stopped window. VolumesReplayed reaches the message. The comment beside the skip was half false and is corrected; the half that still holds — the live unit is the local path's source — is named, and scenario D fingerprints the whole live unit across the operation. Seven red-proofs, each asserted applied and reverted. Two found defects in the tests, not the code: scenario D passed with the unit guard removed because the fingerprint had been narrowed and was blind to the unit root.	2026-08-22 09:43:22 +02:00
admin	f94543ee5c	v0.217.0: prefill from the app's own backup, where-the-data-goes on deploy, bounded inventory fan-out ... Completes R-351 and ships R-352's visibility half. Gates 11/11 OK, suite 28 packages ok, go vet clean, -race clean on the changed package - all run and read BEFORE this commit. PART 2 SCENARIO A - the deploy page prefills the address and data folder from the app's OWN backup. backup.RecordedUnitForStack scans every readable namespace root (the app is NOT installed in this case, so there is no own drive to ask) and reads manifest.json plus the captured compose/app.yaml. Local file reads only: no network, no restic, no restore. RecordedAddress.Known() requires BOTH halves on purpose - an absent SUBDOMAIN makes the live deploy path substitute the CATALOG default (stacks/deploy.go:88-90), and offering that back as "what your backup says" would be a fabricated fact. The prefill is labelled as coming from the backup and stays editable: a memory, not a lock. PART 1 VISIBILITY (R-352) - the deploy page now states where the app's data will live before the button is pressed. Measured 2026-08-21: 13 of 53 catalogue templates declare a storage field; the other 40 have none and their data goes to the system drive, which no screen said. Metadata.HasDeployField answers "does this app have somewhere to PUT a recorded value?" - for the 40-class a recorded placement is a fact to state, never a value written into a field that does not exist. NO PLACEMENT CHANGED. NOTHING MIGRATED. The rest is a filed specification. PART 4 - measured before theorising, on the live off-site target: snapshots --json 2605 ms once; stats 2697 ms PER APP, sequential, 5 app tags => 2605 + 5*2697 = ~16.1 s, matching the reported ten-to-fifteen seconds. The cause is the shape already on file, so the per-app size calls now run concurrently, BOUNDED TO 4. The bound is the safety property, not the speed one: the repository is a Hetzner Storage Box with a session cap, and a refused size call returns SizeBytes 0 - a silent UNDER-REPORT of the customer's data rather than a visible failure. Peak-in-flight is asserted. OffsiteInventoryList had no test at all before this. TEMPLATE SAFETY - every Restore* key is set UNCONDITIONALLY in the deploy handler, because a template doing index/eq against an undefined key errors at RENDER time: green build, green vet, green suite, 500 on the page. Four render tests, one per branch, because the existing deploy render test only renders AutoFields and never reaches these blocks. RED-PROOFS, mutation asserted applied then reverted to 0: A three template guards dropped (count asserted 3) -> the blank form returned P4 inventorySizeConcurrency = 1 -> "peak in flight was 1", elapsed 282ms = sequential DOCS: CHANGELOG v0.217.0 (MinAgent 0.129.0 unchanged), CONTEXT (the restore's own memory + what is next), controller/README.md (Backup System), REUSE.md (4 new rows), REPORT.md overwritten - the previous REPORT preserved to audits/REPORT-v0.216.0-2026-08-14.md first. NOT fixed here, filed as R-353 and named the next session's first item: a restore whose unit carries no db_dumps and no volume_dumps still reports a bare completion.	2026-08-21 21:29:01 +02:00
admin	985388c6e9	R-351: the restore compares where the backup says the data lived; second press cannot start a second run ... Part 3 (not droppable) and the engine half of Part 2. No version bump yet - one bump and one bake at the end of the session. PART 3a - a second press really did start a second run. Established with a test BEFORE any change: both offboxReconstituteHandler and offboxPlaceHandler answered "...elindult" and overwrote the first restore's op/stack. Cause: every restore handler gated on backupMgr.IsRunning() - the CONCURRENCY flag, which the restore goroutine acquires AFTER the handler returns (offbox_reconstitute.go:180, offbox_restore.go:393). Seven sites. The wizard had read the correct flag since v0.154.0 and said so in a comment; the handlers never moved. New Server.restoreOpBlocked() reads BOTH flags - the display flag covers the whole off-box restore, the concurrency flag is the only one the nightly backup holds - and the refusal now names the running app and a route. PART 3b - the page DOES refresh; the defect was the RESULT. backups_shared.html gated the terminal result on a page-local sawRunning flag, so a restore that finished before the page was opened, or inside one 3s poll, was shown to nobody. The 2026-08-21 OpenGist restore took 8.666s and no screen ever said it completed - the answer existed only in docker logs. RestoreOpStatus.LastRecent now carries the server's verdict. The 10-minute window moved to internal/backup as RestoreResultWindow and internal/web's constant is an alias: one expression, two surfaces. Also removed the wizard's self-contradiction, which said the state refreshes automatically AND that you must refresh the page. PART 2 (engine) - every recovery unit manifest has carried drive and namespace_root since schema 1, and NO non-test code read either back. The reconstitution opened the manifest and took only the coherence stamp, then resolved its destination from the live app. A restore into a different destination succeeded silently under a green message. New backup/offbox_placement.go: CheckPlacement (pure, total), PlacementMismatchMessage, recordedPlacementFromScratch. Compared before the safety dump and before the first byte. A mismatch is NAMED and refused; ackPlacementChange lets the customer proceed deliberately - a separate field from confirm=1, because one click must not carry two decisions. An UNKNOWN recording is never a mismatch: refusing on an absence would strand every pre-field unit. The not-installed refusal (R-253) now names the drive the backup recorded. RED-PROOFS, each mutation asserted applied and reverted to 0: B both guards removed (count asserted 2) -> the restore WAS seen starting with no drive attached: no error, full 3.00s run, wrote into /tmp/mutant-destination C Mismatch forced false -> the silent divergent restore returned E Known() forced true -> the fabricated empty prefill appeared D Mismatch forced true -> 8 ordinary reconstitute tests broke, proving reachability both ways Note on D: the existing fixtures write a schema-1 manifest with NO drive, so they are scenario-E shaped. The matching case is covered in the scenario table, not by them. Gates 11/11 OK. Suite 28 packages ok. Hungarian verified as hex, no BOM, no mojibake sentinels. NOT in this commit, still open: Part 2's scenario-A prefill UI, Part 1's deploy-page visibility line, Part 1's specification document, Part 4's measurement.	2026-08-21 21:04:16 +02:00