docs: R-191 closed, R-190 mitigation shipped (mechanism open), R-194 filed
gates / gates (push) Successful in 7s
gates / gates (push) Successful in 7s
R-191: the gate was run first and it mattered — ep0's prune jobs have run daily since 2026-07-27 (18 tasks, all OK). THREE of my own queries said the opposite and all three were broken instruments (worker-type vs worker_type, prune vs prunejob, proxmox-backup vs proxmox-backup-proxy); acting on any would have disabled the only pruning attempt while reporting that nothing prunes. R-190: the row now reflects the MITIGATION, not the cause, and says so. Includes that v0.124.0's transition record failed in production while every unit test passed, and Part 3's single pass — the token-rotation class is real and documented in our own installer but does not fit this box (same token still authenticates, the other three grants survived, no installer run evidenced); ruled out on this box, not fleet-wide. R-194: PVE caches permissions (~40s and ~16min measured), so a single permission read is a lagging indicator and may explain when R-190's refusal surfaced. 07-backup-architecture: the offsite retention row was RIGHT and the configuration disagreed with it — noted, since a doc that states a contract does not enforce it.
This commit is contained in:
@@ -20,36 +20,30 @@ Proven end to end on real hardware.
|
||||
|
||||
## What's broken
|
||||
|
||||
- **Rebuilding a machine silently takes away its off-site app-data backup.** `demo-hp` was rebuilt on
|
||||
3 August and came up without one, and stayed that way for a day. **Fixed on 4 August** — re-issued,
|
||||
the machine picked the new password up in 15 seconds and reattached to the same repository, and you
|
||||
escrowed the key. **The underlying fault is not fixed:** the off-site password is delivered exactly
|
||||
once and a rebuilt machine cannot ask for another, so this will happen again on the next rebuild.
|
||||
The other machine survived the same rebuild only because an unused password happened to be waiting
|
||||
for it. *(R-193)*
|
||||
- **The daily email about it tells you the wrong story**, and the automatic repair that exists for
|
||||
this declines without saying why. The message says the password was never applied; it was, on
|
||||
23 July, and worked for eleven days. *(R-192)*
|
||||
- **The weekly off-site backup reports FAILED although it worked.** It uploads correctly and then
|
||||
trips on a tidy-up step it is deliberately not allowed to perform, so the job ends in an error and
|
||||
you get an email. The backup itself is safe and on the endpoint. Both demo machines do it; one
|
||||
setting per machine fixes it. *(R-191)*
|
||||
- **Rebuilding a machine silently takes away its off-site app-data backup.** Fixed on demo-hp on
|
||||
4 August (re-issued, picked up in 15 seconds, same repository, you escrowed the key), but the
|
||||
underlying fault stands: the password is delivered once and a rebuilt machine cannot ask again.
|
||||
The daily email about it also tells the wrong story. *(R-193, R-192)*
|
||||
- *(fixed 4 Aug)* ~~The weekly off-site backup reports FAILED although it worked.~~ It uploaded fine
|
||||
and then tripped on a tidy-up step it is deliberately not allowed to perform. The machine no longer
|
||||
asks — tidying up is the endpoint's job, and **that was checked first**: the endpoint has been doing
|
||||
it every day since 27 July. Both machines and all future ones. *(R-191)*
|
||||
- **The off-site copy can be erased by the machine that made it.** The credential that writes it can
|
||||
also delete it. A daily snapshot is armed as a stopgap.
|
||||
*(R-95, R-87)*
|
||||
|
||||
## What shipped recently
|
||||
|
||||
- **The on-machine backup copy has now been proved to restore — by the machines themselves.** Both
|
||||
demo machines restored their own on-machine backup into a throwaway machine overnight, booted it,
|
||||
checked it and destroyed it, without being asked: 84 and 109 seconds each. Every restore proof we
|
||||
had before this was of the *off-site* copy; the copy an ordinary recovery would actually use had
|
||||
never been tested on either machine. Both also proved their off-site copy on the same night, one
|
||||
after the other rather than at once, which is the machine deciding for itself what to do first.
|
||||
*(closes the last open half of R-86/R-185)*
|
||||
- **A backup copy the machine was never allowed to read — and could not tell you about**, on both
|
||||
demo machines. The permission was one command; the silence was the real fault, and the machine now
|
||||
checks whether it may read each copy it depends on and says so when it may not. *(R-185)*
|
||||
- **A permission the machine needs can now repair itself — and it tells you it had to.** Yesterday we
|
||||
found a permission that vanished with no reinstall and no recorded action; the machine now notices,
|
||||
restores it through the same controlled helper that grants it in the first place, and **raises an
|
||||
alert saying it was missing and was restored**. The alert is the point: a permission that silently
|
||||
fixes itself is one that can be lost over and over and never investigated. Proven twice on a demo
|
||||
machine by taking the permission away by hand. *(R-190 — the repair; why it vanished is still open)*
|
||||
|
||||
- **The on-machine backup copy has now been proved to restore — by the machines themselves**, both of
|
||||
them, overnight and unasked (84 and 109 seconds). Every restore proof before this was of the
|
||||
off-site copy. *(R-86, R-185)*
|
||||
- **Three ways the alarm system was misreporting its own work — all fixed.** None of them ever risked
|
||||
data. **(1)** When the machine proved a backup restores, that result could vanish if the agent was
|
||||
restarted in the following quarter-hour — and yesterday's change made the gap a week rather than a
|
||||
|
||||
Reference in New Issue
Block a user