985f0ba63c
gates / gates (push) Successful in 30s
R-273's owed guards are both built and closed. R-291 records what CI stopped covering and why, so it can be widened deliberately rather than discovered. R-292 is new and was found by a test failing for the wrong reason: artifact_sha_invalid conflates "version missing", "registry unreachable" and "bad sha" into one message. v0.102.0 works around it by ORDERING -- the probes run first, so an unreachable registry is reported as unreachable -- but the message itself is untouched. CONTEXT gains the rule this session is about: a check and the policy it enforces must read the same number from the same place, or they drift and the drift looks like a defect in something else. Two corollaries, both of which cost something: a bounded check must print what it stopped covering on every run, and an unreadable policy is INCONCLUSIVE rather than unbounded. Stated in the report rather than glossed: Part 4 (finding receipts for the twelve downgraded claims) was NOT done and is a shortfall, not a decision -- splitting it would have produced exactly the half-checked green the exercise exists to prevent. Part 5 was droppable and dropped. The tag-push green is not re-proved tonight and is not claimed; the evidence offered is runs 190 and 216.
216 lines
14 KiB
Markdown
216 lines
14 KiB
Markdown
# STATUS — what works, what's broken, what's next
|
|
|
|
**Updated 2026-08-09 (evening).**
|
|
|
|
> **A view, not a source.** `documentation/backlog/OPEN-ITEMS.md` is the authority; this page restates
|
|
> part of it in plain words, and **nothing may exist only here**. Not `CONTEXT.md`, which is technical
|
|
> state written for Claude Code. **Items, not paragraphs. One screen.** If it does not fit, something
|
|
> belongs in the register instead.
|
|
>
|
|
> *Rebuilt from the register on 2026-08-07, from 258 lines. The old "what shipped recently" log is what
|
|
> the per-repo `CHANGELOG.md` files and the register are for, and is not restated here.*
|
|
|
|
## ⚠ BOTH DEMO MACHINES ARE OFF AND MUTED — unmute them when they are home
|
|
|
|
**Powered down 2026-08-09 14:08 CEST** for the move back from the vacation home. Guests stopped
|
|
cleanly first (no vzdump was running, no locks), then the hosts. Confirmed off at the fabric, not
|
|
merely unreachable: the tailnet is healthy and both peers report *"offline, last seen 1m ago"*.
|
|
|
|
**Both customers are BLOCKED on the hub, deliberately, to stop four false alarms an hour into the
|
|
drive.** Blocking gates every monitor and the notification intake; it does **not** gate config pull or
|
|
report intake, so the boxes come back normally on power-up.
|
|
|
|
> **THE TAIL, and it is the reason this banner exists: while they are blocked, a box that FAILS to
|
|
> come back up is also silent.** When the machines are home and powered on, unblock them and confirm
|
|
> both report:
|
|
>
|
|
> Hub → Customers → **demo-hp** → Unblock, and **demo-felhom** → Unblock.
|
|
>
|
|
> Then check both read ONLINE on Hosts. **Until that is done, the hub cannot tell you either box is
|
|
> in trouble.**
|
|
|
|
## What works
|
|
|
|
A blank machine boots the Felhom disc, installs itself unattended, and is claimed by the customer, who
|
|
sets their own password. They install apps from a catalogue of fifty-three, share files over the home
|
|
network, and open apps from a launcher or a shared link. Backups run on their own to three places — the
|
|
machine's drive, a second drive, and an encrypted off-site copy.
|
|
|
|
**The backup promise is proved, and so is getting the data back yourself.** A machine has been
|
|
destroyed on purpose and its files came back byte for byte identical — four times now. On
|
|
**2026-08-07 the household's own journey passed for the first time**: someone with a browser and
|
|
their recovery code got everything back with **no command line inside the machine at any point**,
|
|
in 72 seconds. The two rough edges that walk found are also gone. *(R-201, R-252, R-253 — closed.)*
|
|
|
|
## The guard is in. An approval that cannot be installed is now refused.
|
|
|
|
**Yesterday morning every install in existence failed for hours, and nothing would have stopped it
|
|
happening again. Now something does.** When you press Save on the Day-0 artifacts, the hub checks —
|
|
before it writes — that each version you are vouching actually has its git label **and** that its file
|
|
can actually be downloaded. If either is missing it refuses and tells you which, for which version,
|
|
and the one command that fixes it. **Hub v0.102.0 is live.**
|
|
|
|
Three details worth your knowing:
|
|
- **It checks the exact file the installer fetches first** — the one whose absence broke Friday — not
|
|
some other file that happens to exist. A test pins that, because probing the wrong file is precisely
|
|
how the failure stayed invisible.
|
|
- **"Could not check" also refuses**, with a different message. Saving with a warning would read as a
|
|
success, and we have the scars. **There is no override**: the registry is on your own server, so if
|
|
it is unreachable the approval can wait.
|
|
- **A second machine now catches it one step earlier** — the agent repo refuses to consider a release
|
|
complete unless its label and its package both exist.
|
|
|
|
## The red repository is green, and the two rules now share one number
|
|
|
|
The tidy-up keeps the newest ten versions; the check demanded every version ever labelled still be
|
|
downloadable. Both are sensible and together impossible, so the red would have returned on your next
|
|
publish. They now read the same number from one file. **What the check no longer covers, plainly: a
|
|
version older than the ten is no longer asserted downloadable** — its label and its config files still
|
|
are — and it prints which ones it dropped on every run so this cannot go quiet.
|
|
|
|
**One thing I could not establish, and I am not guessing.** Who actually deleted the old versions is
|
|
**still unknown**. Gitea keeps no deletion trail: no cleanup rule is configured, the version table has
|
|
no deleted-marker, the activity feed shows no package operation, and the server log no longer reaches
|
|
back that far. I also **withdrew my own claim from yesterday** that the logs showed no deletion — the
|
|
logs did not cover the window, so they never said anything.
|
|
|
|
## The rehearsal finished. The data came back byte for byte; the journey did not.
|
|
|
|
**We wiped a working demo machine and put it back. All four test files returned identical — including
|
|
the two with Hungarian accents, checked as raw bytes, not as text on screen.** The unlock took 21
|
|
seconds and the restore 13. **But it only finished because I could open a terminal twice.** A
|
|
household would have stopped, twice, and the second time the screen would have told them it was easy.
|
|
|
|
**The two walls, both fixed-or-fixable, neither about the data:**
|
|
|
|
- **The install died four steps in** — the agent version you approved had been published as a download
|
|
but never given its version label, and the installer looks it up by that label. **Now unblocked** —
|
|
I pushed the label after checking the published file matched what you vouched. *(R-273 — closed. The
|
|
two guards that would stop it recurring are still owed.)*
|
|
- **A reinstalled machine cannot re-attach its own data drive.** Every route is a dead end, and the
|
|
restore page cheerfully says „**Ez két kattintás**" while pointing at an empty list. The drive is
|
|
fine and the machine can see it — it just is not offered, because the same drive is also the backup
|
|
target. I got past it by typing an internal path no customer could know. **This is the one to fix
|
|
before the tester's visit.** *(R-280)*
|
|
|
|
**Also broken, found on the way:**
|
|
|
|
- **Taking Felhom off a machine leaves the one thing that stops it going back on.** We install a small
|
|
network service at setup; removing Felhom restarts it without its settings, it seizes the port the
|
|
next install needs, and the next install then refuses — appearing to blame the owner's network.
|
|
*(R-272)*
|
|
- **A machine we removed keeps its private line to us open.** *(R-276)*
|
|
- **The hub said nothing at all** while a machine was wiped, rebuilt, re-claimed and had its sealed
|
|
backups opened. No false alarm — but also no word, and the alarm that exists for "someone is opening
|
|
this customer's backups" stayed silent through a real one. *(R-281)*
|
|
- **A rebuilt machine may still come back on software from last week** — narrower than I first wrote:
|
|
the resumed install fetched the right version, but a fresh one takes whatever copy is newest on the
|
|
disk without checking it against what you approved. *(R-274)*
|
|
- **demo-felhom has not had an off-site backup in six days** and is waiting for a recovery code nobody
|
|
has entered. demo-hp, rebuilt the same day, recovered by itself. *(R-278)*
|
|
- **One code, three different names**, and the email points at a page the machine is not showing —
|
|
this cost us a wasted code today. *(R-282, R-283)*
|
|
|
|
## What's broken
|
|
|
|
- **Nothing else new is broken.** The *check* against a fourth secret-in-a-page covers 4 pages of 27, and
|
|
the cheap one covering all of them is blind to the shape that shipped. *(R-255)*
|
|
- **An already-paired box is still told to pair itself**, 25 minutes on. *(R-214, R-235)*
|
|
- **A backup that covered nothing still calls itself „Sikeres".** *(R-240)*
|
|
- **A machine waiting for its recovery code can stop backing up off-site without alarming us.** *(R-243)*
|
|
- **The card offering to reopen set-aside backups promises more than we can deliver.** *(R-202)*
|
|
- **Deleting a customer leaves rows behind** while reporting a clean teardown — no secrets, but it
|
|
accumulates. *(R-244)*
|
|
- **Putting restored files back where they belong is still manual.** *(R-213)*
|
|
|
|
## The rest of what the rehearsal found
|
|
|
|
Sixteen findings in one afternoon, **none of them visible from reading the code** — three sessions of
|
|
review had not seen any.
|
|
|
|
- Removing Felhom leaves five files holding old keys *(R-275)*, and rotating a leaked key does not
|
|
revoke the old one until the service restarts *(R-269)* — the written recipe for it is a step short
|
|
*(R-270)*, and the alarm it raises can never be closed because the fix it recommends is what
|
|
silences the all-clear *(R-271)*.
|
|
- **It caught me being wrong twice, and that matters more than the count.** I told you the fleet's
|
|
off-site backups were down; demo-hp had eighteen snapshots and I had read three misleading screens
|
|
instead of asking the machine *(R-277)*. And I raised a leftover permissions file as a security
|
|
hole, then tested it and refuted myself — it is inert.
|
|
|
|
**What worked, and should not be lost in the count:** the machine came up on its own at the approved
|
|
version; the setup page appeared unprompted, in Hungarian, naming the customer; **the recovery screen
|
|
appeared without being looked for** and said plainly that unlocking changes nothing; the restore told
|
|
the truth about putting files in a checking folder rather than back in place; and no false alarm fired.
|
|
|
|
Full account: `documentation/audits/REHEARSAL-byo-reinstall-2026-08-09.md`.
|
|
|
|
## Three rulings, written down so they stop living in a conversation
|
|
|
|
- **The managed-update floor.** It is deliberately parked, and the trigger to raise it is **the first
|
|
machine that is not ours**; after that it moves with the publish train. Worth knowing alongside it:
|
|
the updater always aims at the floor, never at the newest, so a machine at or above the floor
|
|
updates to nothing. **Correction to the number that was going round: the floor is live at 0.200.0,
|
|
not 0.156.0** — checked twice today, on the hub page and in both machines' own logs.
|
|
- **R-264 is decided.** Build a reader for guest-network health, the staged-update pair, the
|
|
restore-test depth pair, and the two backup-integrity timestamps. Record a stated "no reader
|
|
wanted" for the repaired-recently flag, the tier-applied timestamp, the config fingerprint, the
|
|
per-stack object and the drive-migration marker. Decide the reporting-disabled flag on its own
|
|
merits — if that is a state we support, it must be visible or a staleness alarm will one day fire
|
|
on a machine that is fine. A "no" ends by changing the allowlist reason from *arguably owed* to
|
|
*deliberately not consumed* — not by ripping out an emitter, which is a two-repo change that also
|
|
breaks a shared fixture. **The implementation is its own session.** And **it is twenty facts, not
|
|
twenty-one**.
|
|
- **R-268 is closed** — the leaked key is rotated, and the rotation is proved in both directions
|
|
rather than assumed.
|
|
|
|
## Fixed 2026-08-08 — four things the machine knew and did not say
|
|
|
|
A rebuilt machine can set up its own recovery again *(R-221, agent 0.128.0 — proved on hardware)*; an
|
|
unreadable disk is no longer drawn as a healthy empty one *(R-259)*; a backup tick now answers about
|
|
*that* app *(R-258)*; our own alarm no longer points at a log that may not exist *(R-265)*. Still true
|
|
and not glossed: a failed disk reading still reaches us as "0 of 0 GB" — the quiet direction, it can
|
|
only miss a true alarm, never raise a false one *(R-266)*.
|
|
|
|
## What we're working on
|
|
|
|
- **Widening the check** so a fourth secret-in-a-page is caught by a machine. *(R-255)* · **R-264 is
|
|
now decided** (above); building the readers is a session of its own.
|
|
- **Proving the hub really keeps the old sealed key** when a machine re-seals. *(R-198)* · Still open,
|
|
none urgent: *(R-256, R-257, R-261…R-263, R-266)*
|
|
|
|
## Waiting on you
|
|
|
|
- **Nothing blocking.** The rehearsal is finished and demo-hp is back in service: agent 0.128.0,
|
|
controller 0.210.0, claimed, off-site backups unlocked and intact.
|
|
- **One decision worth taking before the tester comes:** whether to fix the drive wall *(R-280)* now.
|
|
It is the only finding that would stop his visit outright, and it is the difference between "his
|
|
data comes back" and "his data comes back if someone types a path for him."
|
|
- **Two guards are still owed** so the install cannot break the same way twice: refuse to vouch a
|
|
version whose label does not resolve, and check that a published version and its label ship
|
|
together. *(R-273's tail.)*
|
|
|
|
## DooPlex infrastructure — separate from the product
|
|
|
|
*Kept under its own heading rather than dropped: these are real asks that need you, but they concern
|
|
the machine all this is built on, not what a customer receives. Mixing them in is why the page stopped
|
|
being readable.*
|
|
|
|
- **DooPlex's own backup keeps every copy inside the same box, and is silent when it fails.** *(R-232)*
|
|
- **193 old images exist only on this machine**, ~27 GB — clutter, not space. *(R-210)*
|
|
- **The hub password needs rotating** — a diagnostic printed it into a session log; nothing suggests
|
|
anyone else saw it. *(R-132)*
|
|
- **After DooPlex next restarts**, read `/var/log/felhom-store-postboot-check.log` — the second-SSD
|
|
move has never survived a reboot; on PASS, 34 GB comes back. *(R-209a)*
|
|
- **Backup scripts on DooPlex are unversioned host state** *(R-231)*, and the instruction-file
|
|
follow-ups each need a decision rather than an edit *(R-229, R-230)*.
|
|
- **The Configuration page is fixed: 26 s → 0.14 s.** It was never hashing anything — the hashes
|
|
are already stored and simply read. It was making 42 calls one after another. Now they overlap,
|
|
connections are kept, the answer is held for a minute, and the old artifacts are gone. Worst case
|
|
is 5 s, once a minute at most. **Your instinct to prune was right and my measurement said
|
|
otherwise** — trimming to ten of each halved the slow path. *(R-267 — closed.)*
|
|
- **The access token I printed into a log yesterday is rotated**, and I checked it both ways: the old
|
|
one is refused, the new one works, and the machine's own channel is back up. *(R-268 — closed.)*
|
|
- **Our build-check alarm has one gap left.** A run that hangs is now cut off after five minutes and
|
|
the mail says how long it took — but **whether the alarm fires at all when the machinery kills a
|
|
run outright is still unverified**, and we have not claimed otherwise. *(R-265)*
|