docs: R-182 closed, R-90 closed on measurement, R-86 unblocked, ep0 record corrected
gates / gates (push) Successful in 7s
gates / gates (push) Successful in 7s
R-182 CLOSED (controller v0.194.0 + hub v0.90.0/.1), proven live on demo-hp. The hub's notification_log for the run reads: two per-app failures RECORDED, one digest SENT naming both, and the customer channel SKIPPED with operator_only. Against the measured previous behaviour — two failures, one email naming one app, one leaving no trace anywhere. Scenario D proved itself on an event I had not planned: disk_critical alarmed on two filesystems, the second was collapsed by the cooldown, and that collapse is now visible WITH ITS KEY. Yesterday it would have left nothing at all. A gap the spec did not anticipate is recorded with its fix: the per-app event also fires from the periodic sweep, outside any run, so making it record-only would have created a NEW silence. The sweep emits a digest too, with no run_id, so it stays under the ordinary hourly cooldown. ep0: MEASURED on the box — 7757 MB (8 GB), 4 vCPU, and the 4 GiB swapfile SURVIVED the resize and is active (checked, because a resize is a stop/start). The 40 GB local disk is UNCHANGED, so no disk figure was touched anywhere. Five documents corrected — three of which the task's list did not name, found by searching. Two audit/evidence documents ANNOTATED, body untouched: they record what was true when written and that is their value. R-90 CLOSED. R-86 unblocked and re-ranked, stated honestly: 8 GB is comfortable, not unbounded — the original OOM was a 14.46 GB restore — so the restore-test cadence should still be paced, just not by fear of the endpoint. target-selection.md's "D-d did not name ep0 either way" is deliberately left standing. It is the operator's question, not CC's. STATUS.md 127 -> 83 lines, items rather than sentences.
This commit is contained in:
@@ -14,114 +14,70 @@ A blank machine boots the Felhom disc, installs itself unattended, and is claime
|
||||
who sets their own password. They install apps from a catalogue of fifty-three, share files over the
|
||||
home network, and open apps from a launcher or a shared link. Backups run on their own to three
|
||||
places — the machine's drive, a second drive, and an encrypted off-site copy — and a customer can
|
||||
restore files and app data from the drive alone. Proven end to end on real hardware.
|
||||
|
||||
**Apps come back after a power cut.** The machine tells an app the customer switched off from one
|
||||
that simply did not come back, and waits for the system to finish starting before deciding instead of
|
||||
glancing once, five seconds in. Hard-reset the demo box six times in a row: everything came back every
|
||||
time, and an app switched off deliberately stayed off every time.
|
||||
restore files and app data from the drive alone. Apps come back after a power cut: hard-reset the demo
|
||||
box six times, everything returned every time, and an app switched off deliberately stayed off.
|
||||
Proven end to end on real hardware.
|
||||
|
||||
## What's broken
|
||||
|
||||
**The off-site copy can be erased by the machine that made it** — the credential that writes it can
|
||||
also delete it. A daily snapshot is armed as a stopgap, and we have never restored from that copy.
|
||||
*(R-95, R-87)*
|
||||
|
||||
**A full disk tells you about ONE app and silently swallows the rest.** Yesterday this was written
|
||||
down the wrong way round — as *too many* emails. Measuring the receiving end reversed it: of nine
|
||||
refusals the machine reported today, **two emails were sent**. When two apps are refused in the same
|
||||
second you are told about one of them, and the other leaves no trace anywhere — not an email, not
|
||||
even a line in the log saying it was held back. So a second app can be going unbacked-up while you
|
||||
have already been told the problem is handled. It is the same fault we fixed once before for
|
||||
whole-machine backups, in a second place that never opted into the fix. *(R-182)*
|
||||
- **The off-site copy can be erased by the machine that made it.** The credential that writes it can
|
||||
also delete it. A daily snapshot is armed as a stopgap, and we have never restored from that copy.
|
||||
*(R-95, R-87)*
|
||||
- **Nothing else open that affects a customer.** The three faults that were on this list yesterday —
|
||||
the reserve watching the wrong step, the last app whose data was never saved, and the alert that
|
||||
told you about one app and swallowed the rest — are all fixed and proven on a real machine.
|
||||
|
||||
## What shipped recently
|
||||
|
||||
**Pushing the installer no longer publishes it.** The script that runs as root on a brand-new
|
||||
machine was copied from the main branch and served within thirty seconds, so pushing it *was*
|
||||
publishing it, with no staging and no way back but another push. It now comes from a **labelled**
|
||||
version: publishing is moving the label, and undoing it is moving the label back — about half a
|
||||
minute, no deploy. The website is untouched by this and still updates in thirty seconds, because a
|
||||
typo fix must never need a release. Proven by actually doing it: a real push changed nothing that
|
||||
anyone downloads, moving the label published it, moving it back restored the previous bytes exactly.
|
||||
|
||||
**The catch that would have made it cosmetic was found and covered.** While it runs, the installer
|
||||
fetches sixteen more files — not nine, and from the *agent's* repository, not the website's. They now
|
||||
come from the same version of the agent the machine is installing. That closed a real fault nobody
|
||||
had noticed: a new machine was getting the agent's tested program and its untested settings files, in
|
||||
one install, from two different places. *(R-110, R-183)*
|
||||
|
||||
**Releasing the agent now publishes it, in one command.** Putting a built agent where a new machine
|
||||
can download it was a step someone had to remember, and it was forgotten three times in five days —
|
||||
the last time leaving both demo machines running a version nobody could download, so a rebuild would
|
||||
have quietly installed the *older* one and reported success. There is now one command that builds,
|
||||
labels, publishes and then **downloads it back to check** — and a check that refuses to stay quiet if
|
||||
a released version cannot actually be fetched. Proven by making CI fail on purpose and then go green
|
||||
again on the same code. *(R-115)*
|
||||
|
||||
**The backup partition is gone and both demo machines run on the new shape** — wiped, rebuilt and
|
||||
taken through the whole customer journey on 3 August, by two deliberately different routes so the disk
|
||||
shape and the delivery route are both proven. The space a backup can use went from 19 GB to 65 GB on
|
||||
the small machine and 45 GB to 233 GB on the big one. Their previous demo apps and data are gone; that
|
||||
was the point of a wipe, and you approved it. *(R-165, R-178)*
|
||||
|
||||
**What replaced the wall now watches the right moment.** The wall was quietly keeping a runaway
|
||||
backup from eating the space the machine needs to run. As first built, that replacement was checked
|
||||
too late — the big write happened first, unchecked — while still promising your last good copy was
|
||||
untouched. Fixed and proven on 3 August: the machine decides once, per app, **before it writes
|
||||
anything**, and that one answer covers all three steps, so a refused app writes nothing, is not
|
||||
restarted, and the promise is now literally true. It also stopped being blind to size. Nothing is ever
|
||||
deleted to make room. *(R-181)*
|
||||
|
||||
**The last of the three apps that never saved their data is fixed.** Installed nowhere, so nothing was
|
||||
stranded — checked on both demo machines and in the fleet list rather than assumed. Proven by the check
|
||||
that caught it, run in both directions: it clears the fixed version and still convicts the old one.
|
||||
*(R-156)*
|
||||
|
||||
**A filling disk warns the customer before anything breaks, and a failed backup reaches you** — the
|
||||
customer while there is still room to act, naming the drive and the space left; you when one app's
|
||||
backup fails, with the disk figures. The customer is deliberately not told about the second: they can
|
||||
free space, but they can do nothing about a failed backup. Both proven by filling a real disk. There
|
||||
are two rules and not one because the serious warning fired on free space while the disk was only 91%
|
||||
full — a percentage alone would have missed it. *(R-167, R-158)*
|
||||
|
||||
**The checks have two nets and the second emails you.** Every repository has one command that runs all
|
||||
its checks, before every push. That one can be skipped, so the build server runs them again and emails
|
||||
you on failure. It cannot *stop* a change — everything goes straight to the main copy with no review
|
||||
step — but it notices quickly and tells you. *(R-29, R-161, R-168, R-169)*
|
||||
- **A failed backup now tells you about every app, once.** When several apps fail in one night you get
|
||||
**one** email listing all of them, what kind of backup failed for each, why, and how much space is
|
||||
left — instead of an email about one app and silence about the rest. Measured before the fix: nine
|
||||
failures, two emails. Separately, **every failure is now written down the moment it happens**,
|
||||
whether or not it is emailed, and **a message the machine decides not to send now leaves a record
|
||||
saying so**. A machine choosing not to tell you and nothing happening no longer look identical.
|
||||
A run that finds nothing wrong stays silent, and a backup you trigger yourself always reports.
|
||||
*(R-182)*
|
||||
- **Publishing became something you do, not something that happens.** The installer that runs as root
|
||||
on a new machine, and the agent, both moved onto labelled versions with a way back; a check now
|
||||
refuses a release nobody can download. *(R-110, R-115)*
|
||||
- **The reserve now guards the step that fills the disk, and its promise is true.** It decides once
|
||||
per app, before writing anything, and no longer waves an app through at 96% full. *(R-181)*
|
||||
- **The backup partition is gone**; both demo machines were rebuilt on the new shape by two different
|
||||
routes. Backup space went from 19 GB to 65 GB on the small machine, 45 GB to 233 GB on the big one.
|
||||
*(R-165, R-178)*
|
||||
- **The last of the three apps that never saved their data is fixed.** *(R-156)*
|
||||
- **A filling disk warns the customer before anything breaks**, naming the drive and the space left.
|
||||
*(R-167, R-158)*
|
||||
- **The checks have two nets** — one before every push, one on the build server that emails you.
|
||||
*(R-29, R-161, R-168)*
|
||||
|
||||
## What we're working on
|
||||
|
||||
- **Now:** nothing outstanding from today — the reserve, the last unsaved app, and both of your
|
||||
decisions are all built and proven.
|
||||
- **Next:** the alert that tells you about one app and swallows the second *(R-182)*.
|
||||
- **After:** the off-site copy that the machine making it can still erase *(R-95, R-87)*.
|
||||
- **Now:** nothing outstanding. Everything raised in the last three days is closed.
|
||||
- **Next:** restore-tests scheduled around each backup rather than on a free-running timer — now
|
||||
unblocked, because the off-site endpoint has the memory for it. *(R-86)*
|
||||
- **After:** the off-site copy that the machine making it can still erase. *(R-95, R-87)*
|
||||
|
||||
## Waiting on you
|
||||
|
||||
- **A job, not a decision: the hub password needs changing.** A diagnostic command printed it into a
|
||||
session log; nothing suggests anyone else saw it. *(R-132)*
|
||||
- **Nothing else.** Both decisions you took on 3 August are now built and proven. One small question
|
||||
will come back later: the automatic check cannot see which version you have told machines to
|
||||
install, only which ones exist — closing that either needs a password given to the build server or
|
||||
a check inside the hub itself. Filed, not urgent. *(R-184)*
|
||||
- **One small question, not urgent.** The automatic check cannot see which version you have told
|
||||
machines to install, only which ones exist. Closing that needs either a password given to the build
|
||||
server or a check inside the hub itself. *(R-184)*
|
||||
- **One older question still standing:** whether the off-site endpoint counts as protected. It was
|
||||
never named either way, and it is yours to settle. *(target-selection.md)*
|
||||
|
||||
## Changed since last update
|
||||
|
||||
- **2026-08-03** — Publishing became something you do rather than something that happens: the
|
||||
installer and the agent both moved onto labelled versions with a way back, and a check now refuses
|
||||
a release nobody can download. Earlier the same day: the reserve now guards the step that fills the
|
||||
disk and its promise is true, and the last app whose data was never saved is fixed. All proven on
|
||||
real machines, not just in tests.
|
||||
Earlier the same day: both demo machines wiped and rebuilt from the new base image and taken through
|
||||
set-up → install an app → back it up → restore it, with the backup space ceiling gone and measured.
|
||||
|
||||
- **2026-08-02** — The false "host offline" warning is fixed. The hub's database was supposed to be in
|
||||
a mode where reading a page cannot block a machine's status update; a one-word difference meant that
|
||||
setting had **never taken effect**, for the hub's whole life. Fixed and verified live. **Also found:
|
||||
the hub's own database is in no automatic backup** — it holds every machine's emergency password.
|
||||
Filed, not yet fixed.
|
||||
|
||||
- **2026-08-03** — A failed backup now reports every app in one email, and nothing is dropped without
|
||||
a record. You rescaled the off-site endpoint to twice the memory, which closed one item and
|
||||
unblocked another; the documents describing the old machine were corrected, and the audit records
|
||||
of what it used to be were annotated rather than rewritten.
|
||||
- **2026-08-03** — Publishing became deliberate for both the installer and the agent, with a way back
|
||||
and a check that a release can actually be downloaded.
|
||||
- **2026-08-03** — The reserve was fixed to guard the step that fills the disk; the last unsaved app
|
||||
was fixed; both demo machines were rebuilt on the merged disk layout and proven end to end.
|
||||
- **2026-08-02** — Thirteen mechanical checks had built up and nothing ran most of them; two were
|
||||
failing quietly. Fixed. Decided the same day: the 20 GB backup partition goes away; and only this
|
||||
machine and the tester's box are protected, every other box may be broken or reinstalled freely.
|
||||
|
||||
Reference in New Issue
Block a user