Commit Graph

384 Commits

Author SHA1 Message Date
admin 3c49dc8ea4 v0.228.0 — the off-site check reads the data; the debug page stops lying (R-399 + R-400)
gates / gates (push) Successful in 12s
R-399: monitoring.integrity.read_data_subset defaults to 100%. A pack damaged
without changing its size made plain `restic check` report "no errors were found"
on demo-hp 2026-08-30; every read-data form caught it. Cost on that 134 MB store:
35.0s structure vs 39.2s at 100%. "off" (any case) is the off token; empty means
not-configured, therefore the default; a malformed value falls back to the DEFAULT,
never to structure. A completed check over 5 minutes logs a WARN naming the
duration, the depth and R-401 — operator log only, no hub event, no depth change.
The depth is now recorded with the verdict (LastIntegrityDepth; empty = NOT
RECORDED, never "structure").

R-400: 24 debug-page references, 17 dispatched, 7 dead — three of which fetched on
page LOAD, so those panels were permanently blank. backup/crossdrive implemented;
backup/infra, hub/infra-push, dr/infra-status, storage/watchdog-status and both
storage/simulate-* deleted with their panels and JavaScript.
scripts/debug_route_gate.py fails in both directions and is registered after the
seven were resolved. 18 referenced, 18 dispatched, none orphaned.

Corrections: the dead-field warning in report/types.go said the controller runs no
integrity check and the notifiers are called from nowhere — both false since
v0.227.0. controller.yaml.example gains its missing integrity: block.
integrityCheckTimeout's "ships OFF" comment rewritten.
2026-08-31 10:24:29 +02:00
admin 45770f2282 v0.227.1: the damage classifier matched restic's ordinary progress output
gates / gates (push) Successful in 11s
A patch and not a rebuilt 0.227.0: that tag was already running on demo-hp, and
re-pushing changed bytes under a live tag is the :latest hazard with extra steps.

looksLikeRepositoryDamage matched bare "pack ", "tree ", "snapshot ", "blob ". A
HEALTHY restic check prints "check all packs" and "check snapshots, trees and
blobs" -- so any check that failed for a NON-damage reason, a connection dropped
mid-run for instance, would have been classified as a corrupted repository and
told the customer their backups may be damaged. That is the false alarm that
teaches an operator to ignore the true one.

Caught by the NEGATIVE control, built from the real bytes of a real passing
check on demo-hp. The spec made the negative control mandatory and this is what
it was for: a control that has only ever seen the failing case proves nothing.

Signatures are now phrases from restic's own error wording.

Also in this commit: CONTEXT.md records the three rulings (take the flag and
skip, due-ness not a weekday, publish on OffboxReportStatus not the R-331 dead
fields) plus the measurement a future session would otherwise assume wrongly --
THE STRUCTURE CHECK DOES NOT CATCH SILENT CORRUPTION. README documents the job,
the route and the config, and corrects a line that listed four debug backup
routes when only two exist. REUSE gains three rows, including one that records
R-398 was my own mistake so nobody re-files it.
2026-08-30 21:22:23 +02:00
admin 36c13cfa0a v0.226.1: ship the CountsUnknown fix under a NEW tag, not a rebuilt 0.226.0
gates / gates (push) Successful in 11s
0.226.0 was already deployed to demo-hp by hand when the fallback defect was
found. Re-pushing a changed image under a tag that is already running somewhere
is the :latest hazard with extra steps -- two different images, one name, and no
way for a box to tell which it has. So the fix ships as a patch.
2026-08-30 19:51:52 +02:00
admin c0c8fe67bf An unknown drawn as a zero: the defect v0.226.0's own fix introduced
gates / gates (push) Failing after 13s
Writing the REPORT's observation "the no-unit fallback already reports a zero
result, which is honest" exposed that the sentence was FALSE.

A zero UnitRestoreResult is Scenario B's shape. So RestoreFromRecoveryUnit's
fallback to RestoreApp -- which returns only an error, and whose signature is
deliberately out of scope -- would have printed "ez a mentes csak a
beallitasokat tartalmazta, adatot nem" over a restore that may have replayed the
app's entire dataset. That is an unknown drawn as a zero: the exact R-88 failure
direction this whole change exists to remove, re-introduced by the change.

UnitRestoreResult now carries CountsUnknown, the fallback sets it, and there is a
fourth sentence claiming only what is known -- the restore ran, the app is back,
and we cannot say what came back. RestoreApp's signature is untouched.

Pinned by TestUnitRestoreOutcome_NoUnitFallbackSaysUnknownNotEmpty. The A5 seam
test was corrected too: its fixture has no recovery unit, so it exercises exactly
this path and had been asserting the wrong sentence -- it now asserts the
unknown, which is what pins the fallback to it.

IT WAS THE observations GATE REFUSING THE PUSH THAT FORCED THE RE-READ. A gate
written to stop findings dying in an overwritten REPORT.md caught a live defect
instead. Also files R-397 (NotifyIntegrityOK/Failed are dead code AND the
monitoring page advertises a weekly integrity check that does not exist) and
R-398 (resticStep is not a seam, which is why R-358's ordering needed an AST
test) rather than leaving them in a file that is overwritten every session.

REPORT.md is the full run record: baselines re-confirmed, per-test results, the
five red-proofs with their observed output, the live validation with verbatim
Hungarian messages, what was NOT validated and why, teardown across three
layers, and the register 165 -> 167 -> 161.

Green gate clean: 28 packages, rc 0. All 12 controller gates OK.
2026-08-30 19:51:16 +02:00
admin b8af72764d R-353/R-357/R-358/R-360: the restore tells the truth (v0.226.0)
gates / gates (push) Successful in 11s
Four defects on the restore surface, all proven on demo-hp during the 2026-08-21
backup-truth drill, all still in shipped code. They share one acceptance idea: a
restore surface must state what it actually did, and must refuse what it cannot
do.

VERSION NOTE. The task specifying this targeted v0.224.0 against baseline
f8c9390. Both were consumed earlier the same day by R-330 (0.224.0) and R-331
(0.225.0). Drift re-confirmed against live Gitea before the first edit, operator
authorised proceeding, every symbol the spec named re-verified present at the
real baseline e5eee50.

R-353 -- a restore that gave back nothing still said it worked.
RestoreFromRecoveryUnit returned only error, so the surface printed
"<app> visszaallitva (<snapshot>)." -- equally true of a run that returned an
entire dataset and one that returned nothing. The count already existed and was
discarded one line deep: restoreDockerVolumesFrom always returned it, the
wrapper threw it away. Now (UnitRestoreResult, error), carrying replayed counts
AND what the manifest LISTED, because zero-replayed has two causes that are
opposite news. Three cases, three sentences, and EVERY one is a claim about the
BACKUP, never about the app -- this path has no SafetyDump discriminator, and
07-backup-architecture 6.3 records that an absent dump says nothing about the
app (R-361 destroyed canonical .sql files for four months).

R-357 -- the destructive restore had no free-space gate. offbox_reconstitute.go
contained ZERO references to offboxFree; all three existing gates guard
non-destructive paths. The gate now sits before mapOffsiteRestorePaths,
writeSafetyDump and StopStack, so a refusal costs nothing. Position IS the fix,
which is why the test asserts StopStack was never called. No headroom multiplier
(matches PlaceOffsiteRestore; the x1.1 elsewhere predicts a download). Fail
closed on either probe <= 0 -- otherwise `free < need` with need==0 is FALSE and
an unmeasurable scratch sails through: a gate present and inert.

R-358 -- a failed download was offered as a good one. The gate answered "the
directory exists and is non-empty", which is exactly what a part-way restic run
leaves. Now a completion marker written 0600 atomically AFTER restic returns
nil, with any stale one cleared BEFORE it starts; both orders pinned by an AST
test because resticStep is not a seam. Both handlers refuse server-side: the
wizard flags control a button, and a hidden button is not a guard.

SCENARIO F ANSWERED, and worse than the question assumed: a unit-only scratch IS
reachable through the real flow, by the most ordinary route. "Ellenorzo
visszaallitas" (mode=unit, advertised non-destructive) writes the SAME directory
-- offboxRestoreScratchDir ignores `full` and --include limits what restic
extracts, never where -- so a customer who ran the SAFE restore was then offered
the destructive one over a unit-only copy. Filed R-396; the marker closes it.

R-360 -- the delete refused only while a BACKUP ran. IsRunning() is FALSE for the
whole of a verification restore; the five sibling handlers all use
restoreOpBlocked(). Its doc comment claimed it already did this, which is why
nobody looked -- corrected in place.

Red-proofs, each printing the pre-fix behaviour, in CHANGELOG and REPORT. The
first R-357 red-proof exposed a hollow test OF MY OWN and it is recorded rather
than quietly fixed: the fixture refused earlier at the placement stat pre-pass,
so `stops == 0` passed against the pre-fix code. Fixture corrected, assertions
reordered so a removed gate reports the outage rather than "no error returned".

Green gate clean: 28 packages, rc 0. All 12 controller gates OK.
2026-08-30 19:31:31 +02:00
admin e5eee501b5 R-331 (controller half): forward stats_known so the hub can tell empty from unmeasured (v0.225.0)
gates / gates (push) Successful in 12s
The hub's operator Backup card read `Snapshots 0 / Repo Size 0 MB / Integrity
Unknown` for EVERY customer, because it rendered the report's `backup` object --
whose snapshot/size/integrity fields have had NO producer since disk-tier restic
moved to the host agent (slice 8C). buildBackupReport leaves them zero
deliberately and says so. Measured on demo-hp 2026-08-30 while that night's log
said `[offbox] backup OK: 8 app(s) backed up, 67 snapshot(s), 2m14s`.

The live numbers were always in the report's `offsite` object, which the hub
already reads for its Offsite page and its fill/staleness alarms. The hub fix is
to render that -- and that made exactly ONE field mandatory that was not being
forwarded.

snapshot_count:0 means two opposite things: "holds nothing" and "never
measured". R-225 measured that confusion inside this repo (a rebuilt box
rendered 0 pillanatkep over a store really holding snapshot f3d9cd67), and
settings.OffboxTarget.StatsKnown fixed it for the controller's own UI. It was
never put on the wire, so the hub was free to make the identical mistake one
layer up -- and did. OffboxReportStatus.StatsKnown now carries it, omitempty, so
an older controller sends no key and a reader degrades to UNKNOWN, never to
EMPTY. Absence is ignorance, not emptiness.

The four dead BackupReport fields stay on the wire (historical reports in the
hub store must keep parsing) but now carry a warning naming R-331 and pointing
at Offsite. TestBackupReport_DeadFieldsStayZero fails the moment a producer
appears for one -- the prompt to update the hub card in the SAME change rather
than ship a field nothing renders.

RED-PROOF: drop `StatsKnown: t.StatsKnown` -> "a MEASURED empty repository
reported stats_known=<nil>". Tests assert the JSON the hub sees, not the Go
struct: measured-empty and never-measured must differ ON THE WIRE, which is the
entire point of the field.

Green gate clean: 28 packages, rc 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LB8FmJaGd2cyjvy6dbEjpM
2026-08-30 18:38:10 +02:00
admin 45b52b6ed5 R-330: live validation on demo-hp — 3 scans inside the window, 0 events
gates / gates (push) Successful in 12s
v0.224.0 deployed to both demo boxes (both `0.224.0 … (healthy)`). Proven
through POST /api/backup/run, the endpoint the UI button invokes: 8 stacks
stopped and restarted over 87s, three dead-app scans ran INSIDE that window
(16:09:26 docmost, 16:09:56 paperless-ngx, 16:10:26 romm -- the same three apps
that alarmed the night before on 0.223.0), zero app_start_failed pushed.

The scan count is the positive control, not decoration: an absent alarm is
equally consistent with "suppressed correctly" and "the scanner stopped".

A first run is discarded IN THE REPORT rather than quietly dropped -- it fired
52s after a controller restart, inside deadAppBootGrace (90s), where the scan
returns early and could not have alarmed whatever the code did. demo-felhom is
deployed but NOT independently proven and says so: its single app cycles in ~1s,
too fast for any 30s scan to land inside.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LB8FmJaGd2cyjvy6dbEjpM
2026-08-30 18:12:26 +02:00
admin 92cebb8c95 R-330: stop the backup alarming about the apps it is holding down (v0.224.0)
gates / gates (push) Successful in 11s
Measured live on demo-hp 2026-08-30 (controller 0.223.0): the nightly db-dump
and offbox-backup legs stop each stack ~13s to tar its volumes while the
deadapp-check job scans every 30s, so the scan caught whichever stack was
mid-cycle and pushed app_start_failed to the customer. 61 e-mails about apps
that were never broken.

The defect is not a missing mechanism. quiesce/suppress.go solved exactly this
in v0.179.0 and works -- but classifyRunStates read only the quiesce loop's set,
and that loop covers the WHOLE-GUEST backup. The per-app legs stop stacks
through Manager.DumpAppVolumesSafe, which registered with nothing. Two
mechanisms stop apps on purpose; only one told the alarm. Fifth instance of the
"seam built but never wired" class, and the first where the unwired half was a
consumer.

The suppression now rides AppStopGuard, which already brackets every deliberate
stop in the product (Begin before the stop, End after a successful restart) at
all three call sites, and which main.go hands as ONE object to the backup
manager and the exporter. scanDeployedAppRunStates takes the union of both sets.
All three per-app stop paths are covered, not only the reported nightly one.

It cannot latch -- End() runs only on a restart that SUCCEEDED, so unlike the
quiesce loop an open-ended hold is a real hazard here:
  1. ReleaseFailed drops the entry IMMEDIATELY on a restart that broke, wired at
     every failure path, so the app alarms on the next scan;
  2. Begin REPLACES the set (one marker file = one operation);
  3. appStopMaxHold (6h) caps a hold nothing released, logged at WARN.
Grace is 180s, deliberately quiesce's own constant and derivation. Suppression
is NOT persisted: after a crash the guard holds nothing and a down app must
alarm. ReleaseFailed keeps the durable crash marker; a test pins that.

Three companion red-proofs, each printing the pre-fix value (REPORT.md section 5):
  - drop markStopped from Begin      -> "suppressed at stop = map[]"
  - drop ReleaseFailed from the dump -> "map[bookstack:true] after a restart that FAILED"
  - pass nil instead of appStopGuard -> the AST wiring test fails
The third is load-bearing: the component was never the broken part, so a suite
that only injected it would have been green against the shipped defect.

Green gate clean: go build + go vet + go test ./... -- 28 packages, rc 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LB8FmJaGd2cyjvy6dbEjpM
2026-08-30 17:58:15 +02:00
admin 9832760027 v0.223.0: the app-down alarm reached nobody (R-329), and the stop nobody heard (R-386)
gates / gates (push) Successful in 11s
R-329. NotifyAppStartFailures emitted severity "warn". The hub accepts exactly
{info, warning, error, critical} and silently coerces anything else to "info",
which severityNotifies then drops BEFORE both legs. Banner shown, event stored,
POST 200, no mail sent. One word.

This is the second time: DiskAlertKind.Severity emitted "warn" until v0.215.0
and its own comment records that every warning-level disk alert went to nobody.
A comment recorded the lesson and nothing enforced it. The guard is now an AST
walk over the whole controller - grep cannot work here, since "warn" appears
legitimately nine times as a healthcheck status vocabulary.

The sweep found exactly one bad severity. Its limits are stated: the walk cannot
follow a variable, so all six dynamic call sites are registered by name with the
values each can take, and a new one fails the test. Two of the six were found by
the guard, not by the hand sweep before it.

Also pinned: fillwatch.Band.Severity() returns "" for BandOK, which would vanish
the same way. It is unreachable because Check() notifies only on escalation -
but that safety lives in a different function from the one that looks unsafe, so
the test asserts the consequence rather than the mapping.

app_start_failed gains a customer toggle, DEFAULT OFF, per operator ruling. The
operator is mailed either way: processOperator never consults customer prefs.
It is deliberately NOT in operatorOnlyEvents, which would make the toggle a lie.

R-386. classifyRunStates decided "the customer stopped this" from the STATE, so
every stopped stack was assumed deliberate. Measured on demo-hp: privatebin
stopped out of band, nine scans, zero events, zero banner - while the comment
beside it claimed an out-of-band stop still alerts.

DesiredState already records the answer and has exactly one writer. Stopped ->
no alarm; Running -> alarm; absent -> UNKNOWN, keep today's behaviour AND say
so. Absent stays silent deliberately: reading it as "nobody asked" would email
about every app anyone ever stopped, fleet-wide, on the first cycle after
upgrade. The gap is bounded not silent - IntentUnknown is set and the names are
logged at INFO on the heartbeat cadence. failedRestart still lifts a Stopped
intent, or F-CRIT-1 re-opens. No new DesiredState writer.

Two settings toggles each governed two alarms. "Lemez figyelmeztetes (90%+)"
also wrote disk_critical, the drive-is-FAILING alarm. Now four honest toggles;
12 became 15. A no-op save stores the existing slice verbatim, so byte identity
is by construction - without that guard the defaults case reorders, which the
red-proof caught.

Test count 1504 -> 1522. Five red-proofs, five seen failing; one passed first
time and is reported - that mutation was inert, not the test weak.
2026-08-23 11:21:06 +02:00
admin 5da11c4480 v0.222.0: ask whether a supervised member is DEAD before whether one is UNHEALTHY (R-384), and stop promising an undo copy nobody looked for (R-383)
gates / gates (push) Successful in 11s
R-384. aggregateState returned StateUnhealthy the moment unhealthy > 0, and the
R-51 mixed-case block that asks "is a supervised member dead?" sat below it. A
two-container app whose database exits goes unhealthy BECAUSE it cannot reach
that database - so the symptom the dead database causes was what suppressed the
alarm for it. unhealthy is not a down state, so classifyRunStates never marked
the app down and app_start_failed never fired.

Measured live on demo-hp 2026-08-22: bookstack-db stopped at 21:27:01 and the
F-OBS heartbeat printed "0 currently down" throughout. R-51's 18-hour immich
failure, back through a different door.

Two things moved, and either alone leaves the defect standing: the supervised
test is hoisted above the unhealthy/starting/restarting returns, and "some
members are up" now counts ANY member not in the down bucket. The old guard was
running > 0, which made the R-51 block unreachable in exactly the case it was
written for.

IsDownState is byte-identical - unhealthy stays excluded, because an unhealthy
container is running and folding it in reintroduces the flapping that exclusion
exists to stop. No new state was minted. Only the ORDER changed. The priority
comment was rewritten because it asserted an ordering the code no longer has.

Three subtests in TestAggregateState_UnchangedBranches were AMENDED: they
asserted an unhealthy/starting/restarting member beat an exited peer on
unless-stopped, which pinned the defect as settled behaviour. They keep their
intent with the down member given a benign policy.

R-383. The double-failure message said the previous state's backup EXISTS,
built from the returned path without asking the filesystem - and a missing file
is one of the two ways that rollback fails. undoCopyPhrase now describes the
copy from disk: present, partial, missing (still naming where it should be), or
never written. Zero-length counts as missing.

Test count 1494 -> 1504. Four red-proofs planted, four seen failing; the two
halves of R-384 convict independently.
2026-08-23 07:25:00 +02:00
admin da75603553 R-385 record: v0.221.1 gets its own CHANGELOG heading
gates / gates (push) Successful in 11s
0.221.1 was built, baked and vouched on 2026-08-23 with no entry of its own.
The prune-ordering fix (commit 810b18a) was written inside the v0.221.0 entry,
so the newest heading in the record named 0.221.0 while the fleet ran 0.221.1.

The image was never in question - the record was. The reasoning is MOVED, not
rewritten: the paragraph is verbatim from the v0.221.0 entry, which no longer
claims it. The new entry also names the pinning test and its red-proof.

Pushed alone and before anything else this session, so the record is correct
even if the rest of the session does not land.
2026-08-23 07:06:18 +02:00
admin 810b18ab8e R-361 follow-on: the undo-copy prune must run above the already-current check
gates / gates (push) Successful in 11s
Excluding pre-restore-* from db_dumps made that list stable across restores, so
CaptureRecoveryUnit's already-current early return began firing where it never
had - and the prune, which sat after it, stopped running in exactly the case it
exists for. Measured on demo-hp minutes after the change: four undo copies on
disk against a cap of three.

The prune is housekeeping on the dump directory and is independent of whether
the manifest needs rewriting, so it belongs above the check. Pruning cannot
disturb dbDumps, which no longer contains those names.

Pinned by TestR361_UndoCapHoldsWhenTheUnitIsAlreadyCurrent; its red-proof moves
the call back below the return and the cap fails at 5.
2026-08-23 00:02:32 +02:00
admin 968c968559 R-361: the safety dump destroyed the app's own database backup
gates / gates (push) Successful in 11s
writeSafetyDump called DumpOne into the app's OWN unit dir and renamed the result
to pre-restore-* afterwards. DumpOne writes <stack>-<dbtype>.sql - the app's
canonical dump - so every safety dump overwrote the app's real backup and then
moved it away, leaving the app with no database backup until the next nightly
run. A local restore-from-unit in that window tells the customer the app never
had a database.

The comment beside it asserted the rename meant it 'can never overwrite the app's
real dump'. False as written, and believed for four months. Measured live before
the fix: docmost and bookstack each held only pre-restore-* files and no
canonical dump.

DumpOneTo takes the final path and derives its own .tmp from it. DumpOne keeps
its signature and calls it with the canonical name. writeSafetyDump asks for its
own name directly; the rename is gone; the comment now states the invariant and
how it is enforced.

db_dumps no longer lists the undo copies. All three consumers of Manifest.DBDumps
were grepped and named - all inside recovery_unit.go, none reads it for recovery.
The files are neither deleted nor hidden.

Tests 1485 -> 1493. FIVE red-proofs, TWO PASSED first time and both are reported:
the behavioural tests inject the dump seam so a mutation inside DumpOneTo was
invisible, and 1.3 had no test at all. Guards added at the layer each defect
lives in; both mutations then convicted.
2026-08-22 23:39:59 +02:00
admin 1a2405e86f R-379: --clear-restore-hold now states the required restart
gates / gates (push) Successful in 11s
It runs as a second process: it clears settings.json but the running controller
keeps its in-memory copy and goes on refusing. Measured on demo-hp - clear
succeeded, file correct, start button still refused until a restart.

Also records the lost-update window between the two processes, and why clearing
through the running controller (the right shape) needs an operator tier the
controller's HTTP surface does not have.
2026-08-22 18:21:47 +02:00
admin 5b52a5964d R-379 fix: the rollback must re-discover the DB container
gates / gates (push) Successful in 12s
Found by v0.220.0's own live walk on its first real run. writeSafetyDump
captures its DiscoveredDB before the stop; the DB-only start then re-creates the
container with a new id, so the rollback's docker exec hit a dead container and
sat in waitDBReady for 30s. The app was held for an infrastructure reason while
its data was recoverable.

Re-discover and match by {stack, engine} - what reimportDBDumpsFrom already did.
Fail closed when the container cannot be found.

No unit test caught it because they all inject the import seam and never look at
container identity. The new test asserts the identity handed to the import.
2026-08-22 18:14:31 +02:00
admin 2c724c9283 R-379/R-380: put the customer's undo copy back when a database restore fails
gates / gates (push) Successful in 11s
R-379 and R-380 were one failure. Both ended with a half-restored database; the
only difference was whether it looked broken. Postgres emptied and crash-looped;
MariaDB applied part of the dump and reported health=healthy with a zero-row
schema-version table. Measured live on demo-hp 2026-08-22.

The undo copy was already taken and already good - proven by hand that day on
both engines. Nothing in the product could apply it. Now it does, with the same
ImportDump call, before any restart and inside the DB-only window.

The WHOLE undo set, matched on this run's stamp. writeSafetyDump returned one
path for an app with two databases; a rollback on that would restore one and
leave the other half-written.

When the rollback also fails the app is HELD STOPPED (operator ruling): a running
app on a half-written database lets the customer make the damage permanent. Every
start path refuses it - customer button, appstop Recover, boot sweep - via the
shared driveStartGate, checked ABOVE its driveless early return because these
apps have no drive. The marker is ended so nothing auto-restarts it. The row goes
red. Cleared with --clear-restore-hold, an operator CLI route.

--single-transaction is a belt on Postgres only; MariaDB DDL is not transactional
and that is why the rollback is the fix.

R-381: the engine's stderr stops reaching the customer (615 bytes on MariaDB, its
middle rows out of their own database) and starts reaching the operator log,
which never had it.
R-382: the summary log prints the volume count it already held.
Undo copies resolve to their own app, are marked IsUndo, and are capped at 3 per
app, pruned from the capture side. The reported render-as-an-app symptom did NOT
reproduce - the live page was read first and had zero occurrences.

Tests 1468 -> 1483. Eight red-proofs; ONE PASSED and is reported: the R-381
behavioural test injected below ImportDump. A guard at that layer now convicts.
2026-08-22 18:03:18 +02:00
admin 08eb1a6e3a R-356: the off-site restore refused every app that has no data drive
gates / gates (push) Failing after 12s
ReconstituteFromOffsite and PlaceOffsiteRestore both resolved the restore
destination with the RAW HDD_PATH and read an empty answer as "the app is not
installed". For 40 of the 53 catalogue apps that answer is correctly empty and
permanent, so both actions refused forever for a running, healthy app — and told
the customer to reinstall it "in the same place", which those apps never offer.

Separate the two questions. "Installed?" is asked of ListDeployedStacks via a new
Manager.isStackDeployed that fails CLOSED on a nil provider. "Where?" is answered
by GetAppDrivePath — the same resolver CaptureRecoveryUnit wrote the snapshot
with, so the restore aims at the place the backup came from.

The 13 drive apps are unchanged: own drive, mismatch check, ack still required.
A third refusal, with its own sentence, covers installed-but-no-resolvable-root.

Fixtures that marked an app "installed" by giving it an HDD path now state
deployment as its own fact. No assertion weakened.
2026-08-22 13:08:46 +02:00
admin 5ce3a44645 v0.218.0: attribute a DB container by its compose project, and replay volumes on the off-site restore
gates / gates (push) Successful in 11s
R-355 (first, because it is the only one where data can be lost for good). paperless-ngx's
PostgreSQL was dumped into backups/primary/paperless/db-dumps/ — a directory for a stack that
does not exist, on the system drive — while the app's own unit recorded db_dumps: null. The
same misattribution reached writeSafetyDump, so a destructive restore of that app took NO undo
copy and the fail-closed refusal was never reached. Fixed by reading the compose project label,
which is the stack name by construction (compose runs with cmd.Dir set to the stack dir and no
-p). The old derivation stays as the fallback and an unresolvable attribution is now loud.
Catalogue sweep, proven able to convict: one affected app of 53. The fix is in the controller,
not the catalogue.

R-354. ReconstituteFromOffsite skipped every unit placement and the volume archives live inside
the unit, so the off-site restore had no volume leg at all — proven live with planted files:
calibre-web's 1,422,848-byte config archive was in the unit, the snapshot and the checking
folder, and the restore reported success without it. For the 40 of 53 apps that declare no data
drive that archive is the whole dataset. restoreDockerVolumesFrom is the local path's own replay
with an explicit directory: ONE implementation, two callers. Volumes replay before the database
and inside the stopped window. VolumesReplayed reaches the message.

The comment beside the skip was half false and is corrected; the half that still holds — the
live unit is the local path's source — is named, and scenario D fingerprints the whole live unit
across the operation.

Seven red-proofs, each asserted applied and reverted. Two found defects in the tests, not the
code: scenario D passed with the unit guard removed because the fingerprint had been narrowed
and was blind to the unit root.
2026-08-22 09:43:22 +02:00
admin f94543ee5c v0.217.0: prefill from the app's own backup, where-the-data-goes on deploy, bounded inventory fan-out
gates / gates (push) Successful in 10s
Completes R-351 and ships R-352's visibility half. Gates 11/11 OK, suite 28 packages ok,
go vet clean, -race clean on the changed package - all run and read BEFORE this commit.

PART 2 SCENARIO A - the deploy page prefills the address and data folder from the app's OWN
backup. backup.RecordedUnitForStack scans every readable namespace root (the app is NOT
installed in this case, so there is no own drive to ask) and reads manifest.json plus the
captured compose/app.yaml. Local file reads only: no network, no restic, no restore.
RecordedAddress.Known() requires BOTH halves on purpose - an absent SUBDOMAIN makes the live
deploy path substitute the CATALOG default (stacks/deploy.go:88-90), and offering that back as
"what your backup says" would be a fabricated fact. The prefill is labelled as coming from the
backup and stays editable: a memory, not a lock.

PART 1 VISIBILITY (R-352) - the deploy page now states where the app's data will live before
the button is pressed. Measured 2026-08-21: 13 of 53 catalogue templates declare a storage
field; the other 40 have none and their data goes to the system drive, which no screen said.
Metadata.HasDeployField answers "does this app have somewhere to PUT a recorded value?" - for
the 40-class a recorded placement is a fact to state, never a value written into a field that
does not exist. NO PLACEMENT CHANGED. NOTHING MIGRATED. The rest is a filed specification.

PART 4 - measured before theorising, on the live off-site target:
  snapshots --json 2605 ms once; stats 2697 ms PER APP, sequential, 5 app tags
  => 2605 + 5*2697 = ~16.1 s, matching the reported ten-to-fifteen seconds.
The cause is the shape already on file, so the per-app size calls now run concurrently,
BOUNDED TO 4. The bound is the safety property, not the speed one: the repository is a Hetzner
Storage Box with a session cap, and a refused size call returns SizeBytes 0 - a silent
UNDER-REPORT of the customer's data rather than a visible failure. Peak-in-flight is asserted.
OffsiteInventoryList had no test at all before this.

TEMPLATE SAFETY - every Restore* key is set UNCONDITIONALLY in the deploy handler, because a
template doing index/eq against an undefined key errors at RENDER time: green build, green vet,
green suite, 500 on the page. Four render tests, one per branch, because the existing deploy
render test only renders AutoFields and never reaches these blocks.

RED-PROOFS, mutation asserted applied then reverted to 0:
  A   three template guards dropped (count asserted 3) -> the blank form returned
  P4  inventorySizeConcurrency = 1 -> "peak in flight was 1", elapsed 282ms = sequential

DOCS: CHANGELOG v0.217.0 (MinAgent 0.129.0 unchanged), CONTEXT (the restore's own memory +
what is next), controller/README.md (Backup System), REUSE.md (4 new rows), REPORT.md
overwritten - the previous REPORT preserved to audits/REPORT-v0.216.0-2026-08-14.md first.

NOT fixed here, filed as R-353 and named the next session's first item: a restore whose unit
carries no db_dumps and no volume_dumps still reports a bare completion.
2026-08-21 21:29:01 +02:00
admin 90f2545679 fix(disk-health): one physical disk must be evaluated once per run (R-335)
gates / gates (push) Successful in 9s
Found on live hardware two hours after the v0.215.0 deploy, by noticing the
release's own positive observable disagreed with its own persisted artefact:
the check logged '3 disk(s) evaluated' while disk-health-state.json held two
records. demo-hp's c11-scratch and felhom-backup are the same NVMe and share
a durable id, so one disk was walked twice per run.

Not cosmetic. The loop writes a disk's record before the next entry reads it,
so the second copy of an aliased disk consumed the FIRST copy's write as its
prior: the disk sustained against ITSELF and reached Hiba on a first sighting,
defeating truth-table row 6 — the rule that separates a one-hour benign
excursion from a false critical. It would also have emitted two identical
events for one drive. Latent on demo-hp only because all counters are zero.

Each diskKey is now evaluated once per run. Both entries stay marked seen so
neither looks like a disappeared disk, and the card still renders both rows —
the dedup is about state and alerts, not display.

Red-proof run and reverted: deleting the guard makes the first sighting emit
Kind:2 (Hiba-from-sectors) at 8 sectors.
2026-08-14 10:30:22 +02:00
admin 8144a70a72 docs(v0.215.0): CHANGELOG, CONTEXT decisions, README feature, REUSE entries
gates / gates (push) Successful in 10s
- CHANGELOG leads with the severity fix and the live warning-vs-warn proof.
- CONTEXT records the settled decisions so they are not re-litigated: Hiba is
  the label for predicted failure (no fourth word); sustain before count and
  why; the provenance of 64/55/60; phase 2 owns the new SMART attributes
  because they are a wire change under G-1; phase 1 state is one record per
  disk, not a series.
- README documents the 14-row ladder, the persisted state, the hourly cadence
  and the five message shapes.
- REUSE pins the severity wire contract on PushEvent — the defect's real home,
  so the next typo'd severity is caught at the table rather than in production
  — and records priorFor vs cardPriorFor, which differ by one observation and
  make the chip disagree with the email if mixed up.
2026-08-14 08:34:31 +02:00
admin 3ed5e3e770 v0.214.0 — the recovery screen stops hedging about a code it can now check (R-311)
gates / gates (push) Successful in 13s
MinAgent: 0.129.0

What was already right: the screen did not bluntly accuse. R-222/R-226 hedged,
naming both causes and the kept package, and saying it could not tell them apart.
That was honest - and it could not tell them apart because nothing ever looked.
Agent v0.129.0 looks, so the hedge becomes an answer.

New class RecoveryCodeOpensRetained on HTTP 422, gated by
FeatureRetainedRecoveryClass (MinAgent 0.129.0). The gate is SEPARATE from the
R-224 one because the two name different agent versions and a box can sit between
them, where a 422 is a shape we did not design. ClassifyRecoveryFailure therefore
takes both flags; the compiler found every call site.

The message says the code is correct, names the supersession date, says the
earlier package is kept, and says the CURRENT backups are unaffected - the half a
customer will otherwise assume wrong. It promises NO restore: there is no
in-product route to a set-aside store (R-312) and the retained package may itself
predate the repository-password field. It routes to support, which can do it.

The claim guard grew a surface and immediately convicted something. It scanned
templates only, while every recovery message is a Go string in a handler - the
highest-stakes copy in the product, never scanned. It now scans recovery_handlers.go
too, and found a PRE-EXISTING unregistered claim on its first run.

Six handler tests asserting which SENTENCE the customer sees; red-proofs asserted
applied, including: 422 unconditional makes an agent that never looked read as
having looked, and routing 400 to the new class congratulates a mistype.
2026-08-12 18:42:02 +02:00
admin 89712563a0 R-302: the abandon banner promises only what the box can still see is true
gates / gates (push) Successful in 10s
The retrieval clause rendered unconditionally on every page and is false on a
reachable state - the same screen where the orphan card says we cannot tell.

The condition is a fingerprint PINNED at the decision, not a comparison against
the current key. The obvious proxy asks about the wrong key: the set-aside
copies were written under an older key the box no longer has, so on a
twice-rebuilt box the proxy promises about copies nothing can open. Demonstrated
- under the proxy, the replaced-package and legacy cases both flip back to
promising.

The pin is a recorded assumption and says so: nothing on the box records which
key wrote those copies. Empty is not a match. A countdown started before this
carries no pin and takes the cautious branch, not a backfill.

A sweep of all 36 templates found a fourth instance (backups page, same
condition applied) and a fifth (the confirmation screen, correctly left alone -
true at the moment of the decision).

New retrieval_promise_gate registers each claim with a reason rather than
banning a verb: a string ban failed twice, and the honest replacement copy
contains the stem.
2026-08-12 15:27:29 +02:00
admin 68f3e12398 R-299: the orphan card's second promise, and a guard that matched one inflection
gates / gates (push) Successful in 14s
The explanation paragraph - the always-visible half of the card - still ended
'a hozzajuk tartozo helyreallitasi koddal kesobb visszaallithatok lehetnek',
the same unevaluable claim v0.211.0 removed from the confirm block below it.

It survived because the spec called that line accurate, and because the
regression guard asserted the SINGULAR form while the card carried the plural,
which does not contain that substring. The guard now matches the stem, so any
conjugation fails it. The two accurate halves are kept.

Also: the guard's failure message sliced rendered HTML at a byte offset and cut
Hungarian mid-character; it now slices on rune boundaries.
2026-08-12 13:49:15 +02:00
admin 86a78c6767 R-294/R-295: orphan card stops promising restorability; one name per secret
gates / gates (push) Successful in 14s
The orphan card told a customer their set-aside off-site history may be
restorable later with their recovery code. The discriminator lives on the hub
and no wire field carries it, so the box rendering that card cannot evaluate the
promise. Copy replaced per the spec: state what happens, decline what we cannot
know and say why, name a route.

The claim page called the same three-word dashboard code two different names
depending on branch, one of which collides with the ten-word escrow code.
Retired 'Visszaallito kod'; the name is now constant and the sentence changes.
Naming only - a test pins that a reset code is still accepted.

secret_in_markup_gate no longer convicts Go template comments, which are
stripped before render; still convicts a real rendered secret.
2026-08-10 13:53:21 +02:00
admin b762a37097 R-280: attach list from mounted-but-unregistered filesystems; two-clicks promise made conditional
gates / gates (push) Successful in 17s
After a reinstall the data drive could not be re-attached through any dashboard
route: both candidate lists came from the agent's unclaimed-disk scan, and the
rebuilt box's drives are claimed. The restore page said it was two clicks while
pointing at an empty picker.

The attach list now also carries the controller's own mounted-but-unregistered
filesystems. initialize is untouched, so the format wizard's system/backup
protection is unchanged. The 'two clicks' sentence is conditional on the picker
being non-empty, and says something true and actionable when it is not.
2026-08-10 13:41:32 +02:00
admin c732fe1283 v0.210.0 — R-259 and R-258: two pictures that were not true
gates / gates (push) Successful in 18s
Both are one shape: something the box already knows, drawn as its opposite.

R-259 — A DISK WE FAILED TO READ WAS DRAWN AS A HEALTHY EMPTY DISK. readDiskUsage
(internal/system/info_linux.go) logged a statfs failure at DEBUG and returned, leaving the caller's
TotalGB/UsedGB/AvailGB/Percent at zero — and usageColor(0) is "nominal". The dashboard's
most-looked-at meter therefore rendered "0.0 GB / 0.0 GB (0%)" with a 0%-wide bar in the healthy
colour. "We could not look" and "there is plenty of room" were the same picture.

readDiskUsage now returns whether the measurement succeeded; SystemInfo gains DiskKnown and
HDDKnown (HDDConfigured is not a substitute: it says a path was configured, not that reading it
worked); and the template draws NO figure, NO percentage and NO meter fill when unknown, saying
"A tarhely merete most nem olvashato ki." instead. A healthy box is byte-identical, colour band
included.

This session rules the convention (felhom.eu CONTEXT.md S-39): an explicit `...Known bool` companion
beside the figures, checked in the template — the shape Offbox.StatsKnown already uses, whose own
comment says "a 0%-wide bar over an unread store is a picture of emptiness, and a picture is a
claim". Pointers and separate error fields are both legitimate Go, but a codebase with three
dialects cannot be gated (ROADMAP G-3 was blocked on exactly this). Existing call sites NOT
converted.

R-258 — THE PER-APP BACKUP TICK WAS GREEN ON PRESENCE, AND RED ONLY ON A GLOBAL CONDITION.
buildAppBackupRows set Tier1LastStatus from status.LastDBDump.Success, which is the box's single
most recent dump RUN, whichever app it belonged to. An app whose own dump failed showed a tick as
long as some other app dumped successfully afterwards; an app with no database took the nil branch
and went green on the mere existence of a restore point.

appDumpVerdict now reads THIS app's own entries in DBDumpStatus.Results (matched on
DumpResult.DB.StackName, failure = non-nil Error). Three states: any failing database -> error; all
clean -> ok; no result recorded -> NO verdict and no icon, titled "Errol a mentesrol nincs
eredmenyunk." The recovery unit carries no per-run outcome of its own, so green cannot honestly be
derived from presence. The global tier1DBStatus label is untouched — it is correct as a global.

RECENCY IS DELIBERATELY NOT ADDED. A tick over a three-week-old restore point is a real weakness,
but an age threshold means inventing a number and the time is already printed beside the icon.
Recorded as an observation, not changed.

AN EXISTING TEST WAS ASSERTING THE DEFECT AND WAS CORRECTED, NOT DELETED:
TestBuildAppBackupRows_Tier1FromRestorePoints expected "ok" for a status with no LastDBDump at all —
green from nothing but a file's existence. It now expects no verdict; its real subject, the
Tier1LastRun time, is unchanged.

The dashboard test EXTRACTS the meter block from the shipped template rather than copying it: a
copied block drifts, and a drifted copy passes while the page it claims to cover has changed — the
fixture-is-not-the-wire mistake this project has now hit twice.

Six red-proofs across both parts, each with the mutation asserted applied.

No new tag on any declared wire — report/builder.go maps into its own types and is untouched;
wire_contract_gate.py confirmed green.

go build / go vet / go test ./... green (28 packages), controller_gates --fast all OK, both run
separately from this commit.
2026-08-08 16:29:52 +02:00
admin fcffaf573a v0.209.0 — R-247: the box stops saying a false thing about its own recovery package
gates / gates (push) Successful in 17s
The answer was on the wire and was discarded at the boundary, for the third time.

The hub has sent `escrow_stale` in the report ACK since v0.57.0 (json:"escrow_stale,omitempty").
report.EscrowStatus had no field for it, so encoding/json dropped it, and an empty restic_pw_sha256
had exactly one possible reading here: "hash-less supersession".

On demo-hp that reading was false in EVERY clause for four days, and the box told the customer so in
its own words. The hub HAD the hash and was withholding it because the escrow row carries a stale
flag (R-246); there had been no supersession; and the bundle DID cover the password — the hashes
matched exactly.

Fixed by receiving the field. EscrowStatus.Stale decodes, and reconcileEscrowed tells the two
conditions apart: a withheld hash now reports that the hub has flagged the row and is withholding,
that this box therefore cannot verify its bundle either way, and that it is NOT established that the
bundle fails to cover the password. The genuinely hash-less case keeps its wording.

Deliberately NOT changed, and said rather than skipped: the stale verdict itself (the hub's flag is
still the hub's verdict; runs still continue), and the customer-facing Hungarian card copy. Clearing
the wrong flag is an operator act hub-side (R-246); re-wording the card is UI work with its own
review path. This change is the wire and the diagnosis.

Found by felhom.eu/scripts/wire_contract_gate.py (G-1), which was built first and seen failing on 40
fields before anything was fixed, and which now refuses any new field of this shape.

go build / go vet / go test ./... green, run separately from this commit.
2026-08-08 08:46:29 +02:00
admin 27d1165962 v0.208.0 — R-254: the last two secrets leave the page source, plus a gate against a fourth
gates / gates (push) Successful in 17s
Site one. app_info.html rendered {{.InitialCreds.Password}} into a hidden span —
a REAL per-install credential, read live out of the running container, in the
response body of every render. The page now carries the non-secret half plus a
boolean; the value comes from POST /apps/<slug>/initial-credentials/reveal, which
RE-READS the container rather than serving a cached copy (caching it in the
handler would put it back in the body one layer in). no-store, CSRF-covered,
logged as an act. Both buttons go through it. A reveal that cannot read the value
SAYS SO rather than returning an empty string that renders as a blank password.

Site two, established before changing. The hidden input is NOT the defect and was
left alone: it fires only pre-deploy, and README §318 documents why the value must
round-trip — the customer notes the generated secrets down and submitting them
back is what makes the saved value the same one they saw. The defect was the
neighbouring READONLY input, which on an ALREADY-DEPLOYED app rendered the secret
into a page with nothing to submit. Fixed by POST /stacks/<name>/auto-field/reveal,
authorised by requiring a type:secret auto-field of that stack. Both directions
pinned.

The premise that this contradicted a repo rule does not hold: the rule is
CONTEXT.md:2070 'Password fields require explicit input — prevents accidental
empty-password deployments', about EMPTINESS. No line in the repo says 'no silent
auto-fill'.

The gate. scripts/secret_in_markup_gate.py, registered in controller_gates.py,
convicts any template expression that names a secret unless allowlisted with a
reason. Its limits are MEASURED and in its docstring: it catches a launder through
a local variable (the assignment names the secret) but is blind to a secret
arriving under a neutral page-data key — verified both ways. That is the shape of
site two, which this gate would NOT have caught. The runtime body assertion covers
all shapes but only 4 of 27 page templates; the other 23 are R-255, filed rather
than glossed. Two nets, different holes, both named.

Correction to v0.207.0's report: HTML comments do NOT ship in the response body
here — html/template strips them, text/template does not. Measured. A red-proof
planting a secret in a comment therefore correctly does not fail.
2026-08-07 21:20:26 +02:00
admin 8dbbc98ff2 v0.207.0 — R-249: the retrieval passphrase leaves the page body; R-252/R-253: two refusals learn to say what to do
gates / gates (push) Successful in 18s
R-249. settings_security.html rendered the passphrase into a display:none
span behind a Megjelenit button. That toggle stops a browser DRAWING the value
and nothing else — the plaintext was in the response body of every render, so a
curl of the page returned it. Found by exactly that: it landed in a session
transcript while driving the documented rebuild path.

The codebase already stated this rule for the recovery code and this page did not
follow it (escrow_handlers.go: 'reveal (claim XHR only — R is NEVER templated
server-side into HTML)'). The page now carries only HasRetrievalPassword; the
value comes from POST /settings/retrieval-password/reveal — CSRF-covered because
POST, no-store, and LOGGED as an act, which reading it off the markup never was.

The tests assert the RAW RESPONSE BODY. Every test that asked what the customer
sees passed while the bytes carried the secret; that is why this survived.

Census: the render-then-hide pattern appears twice more — app_info.html (a real
per-install app password in a hidden span) and deploy.html. Filed as R-254, NOT
fixed here.

R-252. A rebuilt box keeps its drives but loses their REGISTRATION. The restore
page now states that before the customer presses anything, says the backups and
drives are both still there, and links to Tarhely > Meghajtok. Page and resolver
ask ONE question — HasRestoreDestination() reads the same
GetSchedulableStoragePaths() the scratch resolver reads.

R-253. The list promised 'a visszaallitas elobb ujratelepiti' three lines above a
refusal that fired BECAUSE the app was not installed. The promise was the wrong
half: reconstitution writes to the app's own GetStackHDDPath, which exists only
once the CUSTOMER has chosen a drive at deploy time. Auto-reinstalling would mean
the product making that choice for them. Copy now says to install first and routes
to /stacks/<app>/deploy.

Both notices are conditional — a healthy box renders as before, pinned by a test
that fails if either becomes unconditional.
2026-08-07 18:04:26 +02:00
admin 0a9158d53e docs for v0.206.0: CHANGELOG, CONTEXT, REUSE, README
gates / gates (push) Successful in 19s
CHANGELOG v0.206.0 with the ruling that reversed the fix, the three changes,
the SEC 7.2 staleness decision, Q7's closed trap, and the two bugs the tests
caught rather than review.

CONTEXT carries the three rules this session established, in the form the next
session needs them:
  - a box does not create a repository key while the hub holds a sealed
    package for it;
  - the fact that answers a question must be kept where the question is asked;
  - fix the state, do not remember that it is wrong.

REUSE gains four rows, each carrying the trap rather than just the signature:
the mint guard is a CONJUNCTION and t.Enabled is load-bearing in the derived
predicate; the discriminator ships INERT unless wired in main.go's confirmer
literal; the countdown removes BOTH halves or neither and must be driven by an
injected clock; and the epoch must be synced FIRST and unconditionally or the
falling edge is lost.

README documents the three customer-visible changes and the operator levers.

No version literal was edited: the controller version is ldflags-only.
2026-08-07 12:12:35 +02:00
admin c6b69d888e v0.205.0 — a run that skipped an app the customer selected is not successful (R-234)
gates / gates (push) Successful in 21s
THE VERDICT. The R-203 block already said "a warning beside a success is read as a
success" and applied it to ONE of the two shapes it describes: an app missing a
declared mandatory FOLDER made the run incomplete, while an app skipped ENTIRELY
still reported ok. Both do now. Which skips count, decided by measurement:
selected+deployed with no recovery unit YES; selected but NOT deployed no (named,
with what to do — a box left amber by an app somebody removed is a status nobody
reads); disconnected/decommissioned drive no (own signal); nothing selected no.
LastSuccess and SnapshotCount still record what WAS captured.

THE FILED MECHANISM WAS NOT THE MEASURED CAUSE, and saying so is the point. §3
stated that toggling an app on leaves it without a bundle so the first run skips
it. Measured on demo-hp: the run's own pre-dump phase calls captureAllRecoveryUnits
for every DEPLOYED stack, through admitApp, before the push — a unit moved aside
was RECREATED and the run reported ok. That state does not survive a run.

What actually produced the 2026-08-06 sequence: the manual run was dropped by the
single-flight while an earlier run was still going. runOffboxBackup returned nil,
the handler had already answered "A tavoli mentes elindult", and the card then
showed the PREVIOUS run's green verdict — read as covering the app just selected.
The decision is now taken synchronously in the handler and a dropped request says
so. The nightly path still returns nil on purpose: nobody asked, and it retries.

§7.3 measured before deciding: CaptureRecoveryUnit writes a few KB of compose +
manifest, only ENUMERATES dumps rather than creating them, is idempotent and does
NOT stop the app — and already runs inside the off-site run. So there is no wait to
remove for a deployed app and NOTHING was built.

28 packages ok, 9/9 gates. Four red-proofs, each asserted to have applied. Fixture
note: the shared provider's ListDeployedStacks returned nil, so Scenario A first
passed for the wrong reason; fixed with an opt-in deployed set that defaults to nil.
2026-08-06 21:58:21 +02:00
admin 53e9bf0224 v0.204.0 — the restore list is keyed on the store (R-237); the size gate stops refusing in silence (R-238)
gates / gates (push) Successful in 26s
R-237: /backups/restore listed apps that are CURRENTLY DEPLOYED and CURRENTLY
TOGGLED ON for future off-site backups. A rebuilt box has neither, so a household
that had just lost everything was shown nothing to restore while the repository
held their snapshots — measured live on the R-201 re-walk. To restore an app you
had to select it, to select it you had to have installed it, and to know what to
install you had to see the backup you could not see.

The store is now the source of the list (offsite_restore_list.go), built on the
existing R-193 OffsiteInventoryList. Installed-ness became a property OF a row,
never a filter on it. Every case is answered rather than hidden: a snapshot for an
app that is not installed is offered and says it will reinstall first; an installed
app with no snapshot is shown as having nothing; an unreadable store renders as
UNKNOWN (R-225's rule, one screen over) AND keeps the action, because "we could
not look" is not "there is nothing"; no-target is its own state. The felhom-offbox
and _shares marker tags are excluded from the app list.

R-238 classified as a HARNESS ARTIFACT: mode=full without confirm=1 is step 1 of a
deliberate two-step — it starts no job by design and redirects carrying
&full_prep=<app>, which deriveWizardStep requires to reveal the commit. A driver
that did not carry it forward landed back on the intent step. The operator's
browser run completed the same restore. The wizard's precedence rules were NOT
re-keyed: a stale ?full_prep= must never resurrect a commit button mid-restore.

The residue WAS real and is fixed: neither branch of that step wrote anything to
the log, so a refusal — including by the headroom gate — left no trace on the box.
Both branches now log, and so does the concurrent-op refusal.

resolveWizardApp is removed: it was dead once the gate moved, and its test pinned
the defect's behaviour (an untoggled app refused), which would have read as policy.

28 packages ok, 9/9 gates OK. Three red-proofs, each asserted to have applied.
2026-08-06 16:44:05 +02:00
admin 9dc26459ea v0.203.0: the box collects what the hub staged for it (R-218 consume half) + R-220's message
gates / gates (push) Successful in 10s
R-218's declaration half shipped in v0.201.0 and works. Its consume half never
existed. Reconcile ran exactly twice per process — at start-up and when the
recovery screen drives it — and BOTH fire before the hub has anything staged,
because the hub stages in RESPONSE to the declaration those runs precede.

Measured on the R-201 re-walk: unlock reconcile 11:43:07, hub staged 11:44:57
saying 'next cycle', a full report cycle ran 11:55:46, still unconsumed at
12:06. A guest command line applied it in 18 seconds — everything correct except
the trigger.

Bridge.RetryIfDeclared re-runs the SAME reconcile on a 5-minute tick, driven from
the box's own published declaration (OffboxReportStatus().State) — the very
statement the hub acts on, so the two cannot disagree.

Poll, not an ACK flag, decided on the promise: the no-target message says 'amint
megvannak' (no deadline) and the card says 'within a day'. Five minutes is inside
both by a wide margin and needs no hub change.

It stops by construction — a healthy box does no work and logs nothing — and the
settle gate is deliberately kept via ReconcileWhenSettled.

The marker was investigated and left alone: applied_marker lives in the guest's
DataDir, which a rebuild destroys, so it cannot suppress a legitimate re-run.

R-220's customer half: the refusal no longer tells the customer to choose from a
list that may be empty. It names the rebuild, points at the Meghajtók page, and
promises no outcome.

Red-proofs: remove the retry -> credential uncollected (the dead end reproduced);
drop the stop condition -> a healthy box hammers the hub; call Reconcile instead
of ReconcileWhenSettled -> settle gate bypassed; restore the old sentence -> the
impossible action returns.

28 packages ok, vet clean, all controller gates OK.
2026-08-06 12:56:12 +02:00
admin 66d80efb9f docs: R-168 is CLOSED — the "CI is still owed" sentence was stale (R-229 part 2)
gates / gates (push) Successful in 19s
Corrected in all four instruction files across all four repos. Found while confirming this
session own push by run ID, which is precisely the check that catches it.

In felhom-agent/CLAUDE.md the sentence contradicted the same file release section, which
already said R-168 mails the failure -- a contradiction inside one instruction file, the exact
class the R-229 work exists to find.

REPORT.md deliberately NOT overwritten in the sibling repos: a one-line docs correction must not
destroy the record of their last real implementation.
2026-08-06 11:02:55 +02:00
admin 7db42c5fec docs: CLAUDE.md becomes a core plus path-scoped rules (R-229)
gates / gates (push) Successful in 12s
215 lines -> 110 (92 effective; block-level HTML comments are stripped before
injection and never reach the model, verified empirically on Claude Code 2.1.222
with a control and a treatment run).

Four new .claude/rules/*.md, each with a paths: glob list so it loads only when a
matching file is read: gates, ui-hungarian, backup-paths, agent-coupling.

The ## Layout tree was deleted as derivable; REUSE.md already owns the per-package
seams its annotations stood in for. The host/access table was deleted in favour of
a pointer to documentation/operations/nodes.md -- it carried three defects at once:
demo-felhom given as the LAN fallback address as if it were the route, a pinned
"agent 0.93.0" against the project's own no-versions-in-docs rule, and the claim
that no drill VM was provisioned on demo-hp. Measured live: qm list shows VM 300
drill-r50. felhom-agent/CLAUDE.md was right; this file was wrong.

Kept verbatim: the seven session-critical invariants, the F9 live-validation fence,
the end-of-session checklist.

controller_gates.py registers the shared instructions gate (felhom.eu/scripts/,
never copied here; an absent sibling clone FAILS).

Docs only -- no Go, no version bump, no image, no deploy.
Ledger: felhom.eu/documentation/audits/LEDGER-instruction-trim-2026-08-06.md

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JJc8sAGRWmavP3rMtdpkr2
2026-08-06 09:38:27 +02:00
admin 7534ea203d CHANGELOG: controller v0.202.0 (MinAgent 0.126.0)
gates / gates (push) Successful in 10s
Declares the coupling: an agent below 0.126.0 answers 400 for both a fetch
failure and a wrong code, so FeatureRecoveryFailureClass withholds the refusal
reading and the 400 degrades to the neutral message. The gate blocks nothing —
it only decides whether the customer may be told to check their typing.
2026-08-06 08:19:07 +02:00
admin a3499d1807 v0.201.0 — a correct recovery code is never called wrong again (CAMPAIGN-11) — MinAgent 0.125.0
gates / gates (push) Successful in 9s
R-216: the offsite key recovery is a coupled feature and now says so. featureProbes +
featureMinAgent 0.125.0 + a Supports gate at the unlock entry point, FAILING CLOSED — an
agent that cannot answer is named as such instead of the customer's code being blamed.
Measured live: a 404 from agent 0.120.0 came back as "we did not accept your recovery
code, check that all ten words", in 0.134 s, against a perfect code.

R-218: delete the repo-password short-circuit in needsOffsiteCredential. The declaration
stops when the TIER WORKS, not when a key exists — installing a key is the recovery
screen's whole job, so succeeding at recovery was switching off the mechanism that would
have delivered the coordinates to use it.

R-219: the unlock finishes the job — place the key, bring the tier up, then list. Without
it the promised listing could never render on the shape the screen exists for.

R-217: an unreadable store no longer claims to have opened with unattributable content
(the OffsiteInventory{} zero value). Opened / empty / unreadable are three states.

R-222: a code that is right about a RETAINED earlier package is named, not blamed. States
what the hub knows and promises nothing — no read path exists.

R-215: GET /recovery is gated on the same predicate as the interception.

Five red-proofs, each demonstrated failing and restored.
2026-08-05 17:48:08 +02:00
admin 62b85ecf13 CHANGELOG: controller v0.200.0 (R-193, the recovery screen)
gates / gates (push) Successful in 9s
2026-08-05 12:46:13 +02:00
admin 992803c10b CHANGELOG: controller v0.199.0 (R-204 item 4, box half)
gates / gates (push) Successful in 10s
2026-08-05 10:49:20 +02:00
admin 33fcc502e4 CHANGELOG: controller v0.198.0 (R-204 items 1 and 3)
gates / gates (push) Successful in 10s
2026-08-05 07:17:32 +02:00
admin 58c703bd44 R-203 Part 2: a run that missed a MANDATORY directory is not a successful run (v0.197.0)
gates / gates (push) Successful in 8s
The gap was already detected and warned about, in Hungarian, naming the app and the folders --
that warning is what stopped the R-201 drill. The defect was that the run still reported `ok`
beside it, and a warning standing beside a success is read as a success.

last_status gains "incomplete": minted, because "ok" | "error" | "running" had nothing meaning
"it ran, and this app is not fully protected". NOT "error" -- the rest of the run worked and
what was captured is real, so SnapshotCount and the LastSuccess anchor still record it. Half a
backup is not no backup.

The gaps are now recorded STRUCTURALLY (offboxRunResult.mandatoryGaps), not only as prose, so
the verdict has something to act on. It reaches the operator through the EXISTING per-run digest
(backup_run_failures) rather than a new event type -- a new type is a two-repo change and the
hub drops anything outside allowedEventTypes.

The stat-filter gains the ClassMandatory check Tier 2 already had. It is a NO-OP today
(TierOffsite admits mandatory only), so no customer-visible warning disappears -- demonstrated
by widening the tier filter alone and watching the check hold the line.

ANTICIPATED: calibre-web on demo-hp has exactly this gap, so its off-site status becomes
incomplete the moment this ships. That is correct and is the point.

Red-proofs: my first Scenario-C proof PASSED because the test only reached offboxCaptureSet
while the mutation lives in runOffboxInternal -- a mutation the test cannot observe is not a
red-proof, and the fix was the test. The run-level test now fails under both mutations
(unreachable gap recording; unconditional ok).
2026-08-04 18:32:56 +02:00
admin 1b1366bb6e controller v0.196.0: the recovered key installs itself (R-200 plumbing half) -- MinAgent 0.125.0
gates / gates (push) Successful in 8s
--recover-offsite-install is the sibling of --recover-offsite-check: same fetch/unseal path
through the agent, same STDIN discipline for R, but it PLACES the recovered repository
password via InjectOffboxPassword so a rebuilt box reopens the history it inherited.

Doing this by hand would put the offsite DATA key through a terminal, a clipboard and shell
history. In-process the value goes agent -> this process -> the 0600 file and is rendered
nowhere.

The confirmation is a SECOND invocation: without --confirm-install it prints both hashes and
writes nothing, so the operator sees the comparison before any write is possible.

Three outcomes, named distinctly: installed (no local password -- the rebuilt-box shape),
unchanged (identical key already present, nothing written), refused (a DIFFERENT key present;
installing would clobber the key the current repository is encrypted under, and no force
option is offered). Exit 2 for the refusal, distinct from 1 for a failed step.

Red-proof: removing the confirmation gate makes the dry run write, failing the test. The
R-persistence test carries a positive control -- a planted copy is found, then removed and not
found -- because an absence check is worth only what its sensitivity is.
2026-08-04 14:27:38 +02:00
admin 9640e51321 controller v0.195.0: prove the offsite key comes back (R-200 plumbing half) -- MinAgent 0.125.0
gates / gates (push) Successful in 10s
--recover-offsite-check is a docker exec diagnostic in the shape of --print-reset-code: it
reads the customer's recovery code from STDIN, asks the agent to fetch this host's sealed
bundle and open it, and reports whether the recovered key matches the one on disk BY SHA256.
Two hashes and a verdict; never a password, never R, never a blob.

R comes from stdin and not a flag because a flag value is visible in ps, in shell history, in
a container's command line and in any transcript of the session that ran it.

IT COMPARES; IT DOES NOT INSTALL. The recovered password is never written to
offbox/repo_password -- installing changes a live box on a path nobody has walked, and that
link is next session's, with the drill around it. A test asserts the data dir is byte-unchanged
after a check; its red-proof (adding the install call) fails it.

Exit codes: 0 match, 2 clean MISMATCH, 1 a step failed -- "it failed" and "it worked and
disagreed" must never share a status. A box with no local password reports distinctly: that is
the rebuilt-box shape, where the next step is to install rather than compare.

Nothing customer-reachable ships here: no card, no form, no preview.
2026-08-04 13:42:50 +02:00
admin 88897a224e v0.194.0 — one operator email per backup run, and nothing dropped without a trace (R-182)
gates / gates (push) Successful in 8s
MEASURED, not supposed. On 2026-08-03 nine per-app recovery_unit_capture_failed
events reached the hub and TWO operator emails went out. The hub's operator
cooldown key is customerID:eventType(+tier) and that event carries `app` but no
`tier`, so the key held no app identifier: the first refused app took the hour's
slot and every other app's failure was discarded BEFORE anything was written
down, leaving no row on any channel.

The obvious fix — put `app` in the key — was ruled against: on a full disk it
produces one email per app, the volume problem wearing the correctness problem's
clothes.

internal/backup/runsummary.go: a per-run collector with exactly admissionSet's
lifetime, fed by all three write legs, emitting backup_run_failures ONCE at the
end and only when something failed. A clean run emits nothing.

The per-app event stays and becomes the RECORD — the hub routes it record-only,
stored and logged every time, never competing for an email slot. The record and
the notification are now different things.

Deliberate skips (disconnected, decommissioned) are excluded: they have their
own alert, and a nightly email about an unplugged drive is one the operator
learns to ignore.

A manual run always reports: the digest carries a unique run_id the cooldown
cannot collapse. Someone pressing the button is actively trying to get a backup.

THE PERIODIC SWEEP GETS A DIGEST TOO. With the per-app event now record-only, a
capture failure found between runs would be recorded and never notified — a new
silence introduced while closing one. That path emits a digest with NO run_id,
so the ordinary 1-hour cooldown caps it exactly as before while the mail now
lists every failing app instead of whichever was first.

A refusal is recorded ONCE, where the verdict is taken, not at the three legs
that consult it — R-181's contract is one verdict per app per run. Noting it per
leg listed one refused app three times and produced "2 of 1 apps failed". Found
by the digest's own test, not in review.

Silence is safe because the hub's deadline check raises expected_backup_missed
from report freshness, independently of any mail this box sends
(monitor/deadline.go:396,417). Confirmed, not assumed.

7 new tests, 4 red-proofs. The main.go seam walk did NOT fail on its first
attempt — the AST test walked the backup package and not main.go; the test was
fixed and the mutation re-run rather than the pass recorded.
2026-08-03 13:46:14 +02:00
admin 6c43bf6156 v0.193.1 — the refusal's size estimate is rendered in bytes, not "0.00 GiB" (R-181 follow-on)
gates / gates (push) Successful in 9s
Found by v0.193.0's own live proof run. The estimate was printed fixed to two
decimal GiB, so every app under ~10 MB rendered as "estimated 0.00 GiB write" —
which reads as "no estimate was available" and is the opposite of what happened.
Observed live on demo-hp 08:59:46: opengist's real 178 KB estimate printed as
0.00 GiB.

Shipped in the same session because it is the same defect class R-181 is about:
a message an operator cannot rely on is worse than no message.

The arithmetic is unchanged and still in GiB — the reserve's own unit, so the
comparison against FloorFreeGiB reads directly. Only the rendering moved to
humanizeBytes. estimatedWriteGiB -> estimatedWriteBytes, with the GiB conversion
done once at the point of comparison.
2026-08-03 11:05:02 +02:00
admin fef07c3923 v0.193.0 — the reserve guards the write that fills the disk, and its promise is true (R-181)
gates / gates (push) Successful in 9s
B2's capture floor (v0.192.0) was consulted in exactly ONE place —
captureAllRecoveryUnits, which writes a few KB. The two legs that write the BULK
into the same backups/primary/<app> tree, the DB dump and the volume dump, ran
FIRST and unguarded. Measured live on demo-hp 2026-08-03 06:40:03: opengist's
volume dump wrote 2.0 GB with no check, free fell to 1.0 GB, and the floor then
refused the cheap write it had already lost the argument to. Its refusal message
claimed "the previous unit is untouched" — measured false: that app's tar had
gone 182,272 B -> 2,147,666,432 B under a stale manifest. Sixth entry in
CLAUDE.md's table of shipped guarantees the code did not provide.

Fix: ONE admission verdict per app per run (internal/backup/admission.go), taken
before that app's FIRST write and covering all three legs — they write under one
per-app root, which is why one verdict can honestly cover them.

- Lazy, at the app's first write, NOT once at run start: app A's dump can put app
  B under the reserve, so a run-start verdict reads a disk that no longer exists.
- Remembered for the run, never re-decided between an app's own legs — that is
  the split this closes. Reset per run.
- Placed ahead of DumpAppVolumesSafe, which stops the stack as its first act, so
  a refused app is never bounced. After the volume-less check, which has no write.
- Exactly one operator alert per refused app per run.
- Leg order unchanged: volume dumps still precede the capture.

The floor is now SIZE-AWARE: it asks whether THIS app's write would cross the
reserve, not only whether the filesystem is already below it — which is how an
app was admitted at 96% and then allowed to write 2 GB. Estimate = the app's
previous .sql + .tar on disk. No history -> headroom-only, deliberately, and the
alert says so.

A container-based du per volume was MEASURED and rejected: 66 timed runs on
demo-hp guest 9201, median ~355 ms/volume (341-404) on volumes holding tens of
KB — container start-up, not the walk. Decisive on top: docker run needs the
writable layer, so it can fail under exactly the pressure the reserve handles.

The message was NOT weakened; the behaviour was moved so the wording became true.
It now also names which term bound. Every claim is checked against a sha256
fingerprint of the tree it describes, never against the log line.

Still refuses and never deletes: nothing here is generational.

11 new tests through the production functions. The DB leg cannot run without
Docker, so its gate is pinned by an AST walk of backup.go asserting admitApp
precedes DumpOne (strings.Contains is insufficient — a commented-out call still
contains the string). 4 red-proofs demonstrated failing then restored.
2026-08-03 10:53:48 +02:00
admin 4be6467b50 v0.192.0 — the capture floor replaces the bulkhead (R-165, decision B2)
gates / gates (push) Successful in 8s
Ships BEFORE the disk-layout merge it exists for, and is harmless on a box
that never gets it. The mp1 partition was a BULKHEAD as well as a ceiling:
it kept a runaway capture from filling the space the container runtime
needs, because /var/lib/docker was a different filesystem. After the merge
it is the same one, and a full Docker data-root is a stopped box.

The floor sits in captureAllRecoveryUnits, checked BEFORE anything is
written: below the reserve, that ONE app's capture is refused, its previous
unit is left byte-identical, the R-158 alert fires with the space figures,
and the loop continues.

Two terms whichever binds first (97% used / 1 GiB free) in fillwatch's
shape, deliberately BEYOND its critical band (95% / 2 GiB) so the customer
is always warned before a refusal can happen — a floor that fires before
its own warning is a silent failure wearing a threshold.

Headroom, never unit size: a per-unit cap would be R-163 rebuilt inside one
volume. Refuses, never deletes: nothing here is generational, so pruning
could only destroy a different app's only local copy; pruneStalePrimaryDirs
is an orphan sweep, not retention, and must not be repurposed.

Tests 1184 -> 1191. One fixture strengthened mid-red-proof: the "old 20 G
ceiling is gone" test sat at exactly 20 GB and survived a literal
UsedGB > 20 cap — hollow. Now 120 GB, and the mutation fails it.
2026-08-03 06:30:19 +02:00
admin 9a3c4855d7 v0.191.2 — a quiet fill check now says so (R-167)
gates / gates (push) Successful in 9s
Earned during v0.191.1's own live validation. After the customer had been
warned, a restart produced ZERO fillwatch lines — equally consistent with
'ran and chose silence' and 'never ran'. Proving the checker was alive
needed a deliberate crossing into the critical band.

For an edge-triggered check the quiet run IS the healthy steady state, so
that ambiguity is permanent rather than rare. Check now logs a per-RUN
summary on every run, counting unreadable separately from healthy so a
drive that has quietly gone unreadable cannot read as 'all fine'.
2026-08-02 23:37:04 +02:00
admin 5adae4dad9 v0.191.1 — the fill check also runs at startup (R-167)
gates / gates (push) Successful in 9s
Found live on 9201: neither sched.Daily nor sched.Every fires on
registration, so a box booting with a filesystem already over the line
would stay silent for up to 24h — the R-100 shape, and the same gap the
hub's own checkers avoid by leaving already-breached keys unseeded at init.

The watcher now runs once 90s after startup as well. Safe because the check
is edge-triggered against persisted state: an already-warned filesystem
stays silent. The delay lets mounts settle so a drive still returning reads
as unreadable and is skipped rather than warned about. Pinned by an AST
assertion — the schedule registration alone no longer satisfies the test.
2026-08-02 23:27:35 +02:00