admin
91cabdde1b
R-198 — host_escrow_superseded shipped with `blob` (the K-escrow / PBS datastore key) and
identity_blob was added to host_escrow LATER, never here. The offsite restic REPOSITORY
password lives in identity_blob. So demoteCurrentEscrowTx -- whose own comment calls it "THE
ONE escrow row-copy routine" -- retained the whole-guest key and silently dropped the off-site
data key, which is the secret the retention was built to preserve. And because the copy happens
as the new blob overwrites the old, the destroying act was the ESCROW CEREMONY: the exact thing
a rebuilt box tells its customer to run, on a card promising in Hungarian that the old backups
stay recoverable. Both demo boxes crossed that line on 2026-08-04.
- identity_blob added to the table (CREATE + additive ALTER) and carried in the shared copy
routine, so BOTH callers are fixed at once: re-escrow and host-delete demotion.
- ListSupersededEscrow reads it back; store.HostEscrow gains IdentityBlob.
- CountCurrentEscrowWithIdentity is the census of who the fix protects.
- Nothing is backfillable: pre-v0.93.0 retained rows have no blob and their sources are gone.
- Tests assert the CONSEQUENCE (a retained row can still yield a repo password), which is why
the pre-existing retention test stayed green for two months asserting the mechanism.
R-197 — SaveHostEscrow returns the hash it replaced; the escrow PUT raises
offsite_repo_key_changed (warning, operator-only, edge-triggered) when both hashes are known and
differ. No hash value travels. Severity chosen for the world v0.93.0 creates: with the identity
blob retained, a changed key is "this history now depends on an older recovery code", not a loss.
R-192 (half) — the stuck alert now reports the two shapes it actually covers, burned and
regressed, each stating its own measurement; the regressed text withdraws the Re-issue
recommendation. Every self-heal refusal leaves a notification_log row with its reason. The
guard's logic is unchanged; its 500-oldest-reports scoping stays OPEN and the window is named in
the alert text so the limitation travels with the number. offsite_delivery_stuck and
offsite_credential_restaged are added to operatorOnlyEvents -- neither was registered and neither
has a customerMessages entry, which is not a block.
R-196 — five comments (not the three the spec expected) claimed ReissueCredentials rotates the
restic repo password. It resets the PROVIDER password and cannot touch the repo password, which
is generated on the box. All five corrected; the staleness mark documented as precautionary. The
BEHAVIOUR stays open.
Not in this release: R-199, R-200, R-201 remain open -- the chain that hands the key back is
still unassembled. Part 5 hit its gate; the orphan card is untouched (R-202).
12 KiB
STATUS — what works, what's broken, what's next
Updated 2026-08-04.
A view, not a source.
documentation/backlog/OPEN-ITEMS.mdis the authority on open work; this page restates part of it in plain words, and nothing may exist only here. NotCONTEXT.md, which is technical state written for Claude Code — keep the two separate. Maintenance: update at the end of every session in which something shipped, broke, or was decided. One screen; cut items rather than extend it.
What works right now
A blank machine boots the Felhom disc, installs itself unattended, and is claimed by the customer, who sets their own password. They install apps from a catalogue of fifty-three, share files over the home network, and open apps from a launcher or a shared link. Backups run on their own to three places — the machine's drive, a second drive, and an encrypted off-site copy — and a customer can restore files and app data from the drive alone. Apps come back after a power cut: hard-reset the demo box six times, everything returned every time, and an app switched off deliberately stayed off. Proven end to end on real hardware.
What's broken
- Rebuilding a machine still throws away its off-site backup HISTORY. The machine invents the key that encrypts its own off-site backups, and a rebuilt machine invents a brand-new one, so the old backups can no longer be opened by the machine that wrote them. Both demo machines did this on 3–4 August — demo-hp lost access to 15 backups (41 MB), demo-felhom to 36 (1.14 GB), and the second one happened in silence for thirteen hours. From today that silence is fixed (below) and the old key survives the ceremony (below), but the rebuild itself still starts a fresh history, and getting the old one back is a path nobody has walked. (R-193)
- (fixed 4 Aug)
The safety net we believed was under all of this is not there.The central hub was keeping the wrong key, and the fix shipped the same day. When a machine re-seals its backups under a new recovery code, the hub keeps the old sealed package — but it was keeping the one for the whole-machine backups and not the one for the off-site file backups, which is the whole point. And because the swap happens in one motion, the act that destroyed the old key was the recovery-code ceremony itself — the exact thing a rebuilt machine asks its customer to do, on a screen promising the old backups stay recoverable. Both keys are now kept. What this cannot undo: anything superseded before today is gone for good, which includes both demo machines' pre-4-August keys — so those 51 orphaned backups were beyond reach even if the recovery codes had been kept. (R-198) - Nothing in the recovery path has ever been performed, and that has not changed today. Making a recovery code is proven, live, with a real customer. Using one is not. No sealed package has ever been handed back to a machine, no recovered key has ever been put back, no old backup store has ever been reopened. Today's work makes the key survive; it does not build the path that hands it back. That is what the proof exercise is for. (R-199, R-200, R-201)
- One screen still tells the customer something we cannot yet promise. The „elárvult tároló" card says the old backups may later be restorable with the matching recovery code. From today that is true for machines that re-seal from now on and false for anything already orphaned — and the machine cannot tell which case it is in without the recovery path above. We deliberately did not patch the sentence: a conditional promise that can still be wrong is worse on that screen than a vague one. (R-202)
- (fixed 4 Aug)
The weekly off-site backup reports FAILED although it worked.It uploaded fine and then tripped on a tidy-up step it is deliberately not allowed to perform. The machine no longer asks — tidying up is the endpoint's job, and that was checked first: the endpoint has been doing it every day since 27 July. Both machines and all future ones. (R-191) - The off-site copy can be erased by the machine that made it. The credential that writes it can also delete it. A daily snapshot is armed as a stopgap. (R-95, R-87)
What shipped recently
-
The daily error email about David is gone. David's record was created when he said he would try Felhom; no machine was ever attached to it, so nothing has ever been expected of it — yet every morning at five the system reported that a database backup had not happened. A customer with no machine ever attached now produces no alarm. A machine that was attached and went quiet still does, because that is a real fault, and there is a test whose whole job is to make sure a future change cannot take that away. David's record was not touched — the record was right; the alarm was wrong. The first quiet morning will be 5 August, so this is not confirmed yet. (R-195)
-
A permission the machine needs can now repair itself — and it tells you it had to. Yesterday we found a permission that vanished with no reinstall and no recorded action; the machine now notices, restores it through the same controlled helper that grants it in the first place, and raises an alert saying it was missing and was restored. The alert is the point: a permission that silently fixes itself is one that can be lost over and over and never investigated. Proven twice on a demo machine by taking the permission away by hand. (R-190 — the repair; why it vanished is still open)
-
The on-machine backup copy has now been proved to restore — by the machines themselves, both of them, overnight and unasked (84 and 109 seconds). Every restore proof before this was of the off-site copy. (R-86, R-185)
-
Three ways the alarm system was misreporting its own work — all fixed. None of them ever risked data. (1) When the machine proved a backup restores, that result could vanish if the agent was restarted in the following quarter-hour — and yesterday's change made the gap a week rather than a day, because the machine correctly refuses to re-prove an archive it has already proven. It is now written to disk with the result and survives. This was caught happening, not predicted: a real 14.5 GB off-site restore passed and left no record at all. (2) Every release had about a fifty-fifty chance of emailing you a failure for a release that worked; the version tag is now published after the binary, and a new check catches the opposite mistake so nothing is traded away. (3) A released binary can now be rebuilt by anyone and checked against the fingerprint you approve — until today, rebuilding produced different bytes. (R-189, R-188, R-186)
What we're working on
- Now: the proof exercise — wipe a demo machine and recover it with a saved recovery code. It is designed and waiting on your go-ahead. Both honesty fixes shipped today: a changed backup key now raises an alarm on the day, and the email that stated the opposite of what it measured now describes what it actually saw.
- Also tomorrow: confirming what the 04:15 off-site run does on both machines. We expect it to refuse and ask, rather than start fresh — but that is a prediction from the code, not something anyone has watched happen.
- Next: proving the off-site app-data copy can actually be restored — the one tier nothing tests unattended. Most of the machinery it needed arrived with the restore-test change below. (R-87)
- After: the off-site copy that the machine making it can still erase. (R-95)
Waiting on you
- (decided 4 Aug — recorded here until the work it unblocks is done) You chose not to keep a copy of the backup key on the Proxmox host. That is a coherent choice and it has a consequence worth restating: the customer's own recovery code is now the only route back from a rebuild, so the sealed package the hub keeps is the single copy. Today's fix is what makes that true rather than wishful — until this morning the ceremony destroyed it. (R-193)
- Your ruling about the recovery screen has been priced, and it can be built. You said: a freshly installed machine that finds a sealed package waiting should say so loudly, offer the customer a box to type their recovery code into, and show what would come back before doing anything. All of it is buildable, and one part is already free — the hub is already telling every machine, on every check-in, that a sealed package exists and which key it covers, and the machine currently throws that message away. Showing a preview is also cheap: listing what is in an off-site store reads it without writing to it, so the customer can see how many backups, from when, and for which apps before committing. The real work is one new connection: the unsealing has to happen in the part that runs on the Proxmox host, because the customer-facing part cannot do it — the same crossing your recovery-code ceremony already makes in the other direction. One thing for you to weigh, which we deliberately did not decide: a screen that takes a recovery code and then shows what is in a backup store is reachable by anyone with the household's dashboard password, and the preview reveals backup dates and app names. (R-193)
- Should we run the proof? Nothing about recovery has ever actually been done, so we designed the exercise: on the spare demo machine, make a recovery code and keep it, put a marked file into an app, back it up off-site, wipe the machine, reinstall, recover with the saved code, and check the marked file comes back byte for byte. About half a day, of which under an hour needs you. The failure we are watching for is precise: if the recovered machine reports one backup instead of the ones we put there, it started a fresh history and the proof failed — "the store opened" is not good enough. It costs that machine's contents if it goes wrong, which is acceptable there. You said yes, after R-198 — and R-198 shipped this morning, so this is ready to run on your word. (R-201)
- The orphaned backups on the storage box — you said delete, and it is still owed. About 1.2 GB across the two demo machines, in set-aside stores nobody can open and nothing prunes. Deliberately not done today: erasing data on the storage endpoint is not something that should ride along with a database change, so it wants its own session. (R-193)
- A job, not a decision: the hub password needs changing. A diagnostic command printed it into a session log; nothing suggests anyone else saw it. (R-132)
- One small question, not urgent. The automatic check cannot see which version you have told machines to install, only which ones exist. Closing that needs either a password given to the build server or a check inside the hub itself. (R-184)
- Nothing else.
Changed since last update
- 2026-08-04 (latest) — Fixed the worst of it. The hub now keeps the off-site backup key when a machine re-seals, instead of only the whole-machine one — four lines of database change that stop the recovery ceremony from destroying the thing it exists to protect. A changed backup key now raises an alarm the same day. The daily email that described the opposite of what it measured now states what it saw, and a refusal to auto-repair leaves a record instead of silence. (R-198, R-197, R-192, R-196)
- 2026-08-04 (later) — Established what a machine rebuild actually destroys: not the storage password we had been re-issuing, but the key that encrypts the backups. Both demo machines lost access to their off-site history, one of them silently. A decision is now waiting on you. The daily false alarm about David is fixed. (R-193, R-195)
- 2026-08-04 — Both demo machines proved their on-machine backup restores, on their own, overnight — the copy an ordinary recovery uses, never tested until now. Found while checking: the weekly off-site backup reports failure after a successful upload. (R-185, R-191)
- 2026-08-03 — Fixed three ways the alarm system misreported itself: a proof of a working backup that could vanish on a restart (seen happening), a release that emailed a failure for a release that worked, and a released binary nobody could rebuild and check. (R-189, R-188, R-186)