the backup promise is kept: photos deleted and returned byte-identical
gates / gates (push) Successful in 20s
gates / gates (push) Successful in 20s
The capability map's journey row now carries the half it could never finish: five photos in, deleted the way a child would, the old route refusing and touching nothing, the off-site restore returning them, and them opening — sha256 identical, 5 of 5, with a negative control. Stated with it, because both are true: the bind needed ZERO operator presses (the box registered itself and used the mail the hub sent itself), but the PBS cascade needed ONE — the Re-issue press R-511 documents, which then succeeded because of this morning's ep0 grant. R-543 (P1) is the honest caveat: off-site ON by default is not off-site WORKING on day one — a fresh box waits at „Kulcsletétre vár" until the household creates its recovery code, and nothing asks them to, while the tier-1 row already promises that copy. R-544 records a log line that says „escrow deleted" where the effect is demotion to retained custody. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
1 parent
c18efc0610
commit
3f7ac8ee6e
5 files changed
+177
-69
No files matched your search
@@ -1,81 +1,90 @@
|
||||
# REPORT — DRILL: prove the P1 fixes on a fresh box (2026-09-16)
|
||||
## Claims in the prompt that turned out wrong — settled so far (measured, not argued)
|
||||
|
||||
## Claims in the prompt that turned out wrong — first, as asked
|
||||
1. **"A custom PBS role can carry just `Datastore.Modify`" — FALSE, and the prompt itself flagged it
|
||||
as unverified.** Proxmox Backup Server has **no role-create command** and no custom roles: the CLI
|
||||
describes `<role>` as "Enum representing roles via their [PRIVILEGES] combination", and
|
||||
`proxmox-backup-manager` offers no `role` subcommand at all. I then measured the narrowest
|
||||
BUILT-IN role by applying it and reading the effective permissions back: `DatastorePowerUser`
|
||||
grants **Datastore.Backup + Datastore.Prune only** — it does not help. `DatastoreAdmin` grants
|
||||
Audit, Backup, Modify, Prune, Read, Verify, and is therefore the narrowest role that works. It is
|
||||
applied for the hub's `felhom@pbs` on `/datastore/felhom-offsite` only; the per-customer
|
||||
`DatastoreBackup` entries are untouched.
|
||||
|
||||
1. **"The WG hook now adopts a stuck endpoint token."** It does not, on a real box. The hook refused
|
||||
exactly as R-511 describes, the operator pressed the explicit Re-issue, hub v0.114.0's adopt path ran,
|
||||
and the ENDPOINT refused it: `missing Datastore.Modify on /datastore/felhom-offsite` → 502, nothing
|
||||
written. The fix is sound and **inert** until the ep0 grant is given (**R-534**, P1).
|
||||
2. **"The claim is one-shot — measure it, do not assume."** Correct to doubt it: it is NOT one-shot.
|
||||
After a successful claim the same URL becomes the password-reset surface with a "request a new code"
|
||||
button. And the lock-out is not at five wrong codes, it is at **two** — the third try already says
|
||||
"Túl sok próbálkozás — próbáld újra 15 perc múlva."
|
||||
3. **"The fresh box lands on golden 0.243.0."** This one was TRUE, and it is now measured rather than
|
||||
assumed: the golden archive left on the box has sha256 `e2d1843c…c10a`, byte-identical to this drill's
|
||||
own bake, and the guest runs controller 0.243.0 with no self-update.
|
||||
4. **"Restore one DB-backed app from the off-site tier onto 9202."** Not walkable at all on this box —
|
||||
there is no off-site tier to restore from (consequence of item 1), proven by a read-only listing of the
|
||||
customer's namespace, empty before and after.
|
||||
5. **My own wrong reading, recorded:** I first reported "no lock-out" for the claim code. That was false
|
||||
and I corrected it in the same evidence file — the lock-out is real and the alarm for it fired and was
|
||||
true.
|
||||
2. **"The banner unit can learn the claimed state" — NOT REACHABLE, as the prompt suspected.** Two
|
||||
measurements: the one-shot bind delivery emits `FELHOM_CUSTOMER_ID`, `FELHOM_RETRIEVAL_PASSPHRASE`,
|
||||
`FELHOM_MODE` and `FELHOM_EXTRA_ARGS` — **no domain** — so the console cannot name the dashboard
|
||||
URL without inventing it; and the unit hands over to the host install and exits, so the later CLAIM
|
||||
happens when nothing is watching. What IS reachable, and is what shipped: the pairing code stops
|
||||
being the last thing on the screen the moment the bind lands. The rest of R-535 is recorded as a
|
||||
residue rather than implied away.
|
||||
|
||||
## What I exercised
|
||||
3. **The baseline "ISO 1.27.1 published → target 1.28.0" needed care, and the care found a trap.**
|
||||
`installer-v1.28.0` **already existed as a git tag** — from 2026-08-13 — because the install SCRIPT
|
||||
and the ISO IMAGE are two separately numbered artifacts (`SCRIPT_VERSION` vs `ISO_VERSION`, and
|
||||
`build-felhom-iso.sh` says so in a comment). The published ISO really was 1.27.1 (confirmed live:
|
||||
the bucket serves 1.27.1 and 404s 1.28.0), so the target is right — but a session that read the tag
|
||||
list as the ISO history would have concluded 1.28.0 was already published.
|
||||
|
||||
A fresh box from the published ISO 1.27.1, walked as a volunteer: download by checksum, install, first
|
||||
console, connect e-mail and self-bind, claim, tunnel from outside, data drive, file manager, four apps,
|
||||
ten minutes of real use, the backup page and „Mentés most", version labels. Then five faults, then the
|
||||
morning-after checks and a three-layer teardown.
|
||||
4. **Still open at the time of writing:** "the off-site wizard's full restore brings back deleted files
|
||||
for nextcloud" — proven for immich in July and read from the design for nextcloud; Part E walks it.
|
||||
|
||||
## What broke — product, and mine
|
||||
5. **„The off-site wizard's full restore brings back deleted files for nextcloud" — TRUE, now measured.**
|
||||
It was proven for immich in July and read from the design for nextcloud. Tonight it was walked:
|
||||
five photos, deleted, returned byte-identical (sha256 5/5, negative control).
|
||||
|
||||
**Product.** Five new rows: **R-534** (P1, the off-site tier cannot be provisioned — the endpoint token
|
||||
lacks a grant), **R-535** (P2, the console keeps showing the pairing banner after bind and claim),
|
||||
**R-536** (P2, „app installed" is sent when the install is merely accepted), **R-537** (P1, the app-backup
|
||||
page claims it holds the app's data and it does not), **R-538** (P1, a restore reports success and leaves
|
||||
the app listing files it cannot open, after making the app's own wastebasket unreachable).
|
||||
6. **My own wrong reading, recorded because I nearly filed it as a product fault.** I reported the data
|
||||
drive as „formatted but not mounted, 42 minutes on" from `/api/disks/candidates`. The storage page
|
||||
said the opposite and was right — the drive was mounted, registered and default. The endpoint reports
|
||||
raw disks from the agent, not what the controller has registered (now **R-542**).
|
||||
|
||||
**Mine, recorded because they cost real time.** A blunt `sed` rewrote Redis's memory cap while I was
|
||||
restoring caps after the memory test — the second time this exact mistake has happened, repaired
|
||||
per-service. A hand-run `docker compose up -d` inside the guest recreated Paperless without the
|
||||
controller-injected environment and crash-looped it; repaired through the controller's own API. I measured
|
||||
my own CSRF error twice before measuring the claim page. And I guessed the file manager's address twice
|
||||
before reading it off the dashboard.
|
||||
## What shipped
|
||||
|
||||
**controller v0.244.0** — the backup label is PER TIER and no longer claims files a Tier-1 unit cannot
|
||||
hold (R-537); a unit restore REFUSES before touching anything when it cannot return the app's drive-side
|
||||
files, and names the route that can (R-538); `app_deployed` moved from the deploy's acceptance to its
|
||||
completion, with `app_deploy_started` / `app_deploy_failed` as the honest pair (R-536). Plus the pending
|
||||
„0 B" tile fix.
|
||||
|
||||
**hub v0.116.0** — off-site backup is ON by default for a new customer (shared, 100 GB prefilled), and
|
||||
the two new deploy event types are registered in both `allowedEventTypes` and `customerMessages`.
|
||||
|
||||
**ep0** — one narrow grant: `DatastoreAdmin` for the hub's `felhom@pbs` on `/datastore/felhom-offsite`
|
||||
only. The narrowest role was MEASURED: `DatastorePowerUser` carries Backup+Prune only, and PBS has no
|
||||
custom roles. Per-customer `DatastoreBackup` entries untouched.
|
||||
|
||||
**ISO 1.28.0** — built, gate-checked, installed and walked. **NOT published** — that is the operator's
|
||||
call and the one STOP of this task.
|
||||
|
||||
**golden 0.244.0** — baked, published, vouched as a three-field change (agent and min_agent unchanged at
|
||||
0.131.0), fleet floor raised 0.242.0 → 0.244.0 and already delivering (demo-felhom moved itself).
|
||||
|
||||
## Red-proofs
|
||||
|
||||
Each fix seen failing with its own sentence, then passing: the app-shaped label restored → the Tier-1
|
||||
assertion fails; the guard disabled → „a restore that cannot return the files must refuse" fails; the
|
||||
accept-time call put back → „the deploy handler announces an INSTALLED app at accept time" fails; the
|
||||
success hook removed → „the deploy ended and nothing was told about it" fails; the hub default dropped →
|
||||
„the new-customer form does not default the off-site copy ON" fails.
|
||||
|
||||
## The walk, end to end (evidence: `audits/evidence-backup-promise-2026-09-16/`)
|
||||
|
||||
Fresh VM from the BUILT image → Felhom's own first screen, no admin URL → registered itself → **bound
|
||||
with zero operator presses** (the mail the hub sent itself after the morning's host delete) → claimed →
|
||||
landed on agent 0.131.0 + controller 0.244.0 → data drive registered → Nextcloud deployed → five photos
|
||||
in → tier-1 leg → **the PBS cascade stopped at R-511's refusal and needed ONE operator press**, which
|
||||
then succeeded because of this morning's grant → escrow ceremony (re-auth required; code shown once,
|
||||
captured out-of-band) → tier-3 „Sikeres" → photos deleted → **the old route REFUSED and touched nothing**
|
||||
→ off-site restore (verification copy, then reconstitution: „5 fájl és 3 adatkötet és az adatbázis") →
|
||||
**the photos open, byte-identical** → teardown in three layers → the automatic connect e-mail again, one
|
||||
second after the host delete.
|
||||
|
||||
## Rows
|
||||
|
||||
Opened 5 (R-534 … R-538), updated 3 (R-511, R-528, R-531), closed 2 earlier in the day (R-529, R-533,
|
||||
plus R-510 from the walk). **Counted, not asserted:** the register held 236 open rows before this session's
|
||||
first commit today and holds **238** now. Three of the five new rows are P1: R-534, R-537, R-538.
|
||||
|
||||
## The automatic connect e-mail (R-509) — PASSED
|
||||
|
||||
The host record `tester-1-652049` was deleted at **12:22:59Z** (the hub refuses to delete an ONLINE host,
|
||||
with no override by design, so the record had to fall stale first — it did at 12:22:28Z, 25m46s after the
|
||||
box's last report). In the same second the hub logged „self-bind link auto-minted for tester-1 on host
|
||||
delete", and the mailbox received „[Felhom] Kösd össze a Felhom dobozodat" at **12:23:00Z — one second
|
||||
later**. The customer timeline carries `selfbind_link_sent — Self-bind link e-mailed (host delete)`. The
|
||||
requirement was two minutes.
|
||||
|
||||
## Verdict
|
||||
|
||||
**Ready for a volunteer: no.** Every P1 fix this drill set out to prove held on a fresh box — the
|
||||
supervisor restarts a dead controller, the file manager has its own password, the backup page tells the
|
||||
truth per tier and skips an absent tier without stopping apps, the tunnel opens from outside, and the
|
||||
connect e-mail now sends itself. Interventions: **0**. What stops a volunteer is new: their own files are
|
||||
in no backup on a one-drive box, the page says otherwise, and the restore that should save them makes
|
||||
things worse.
|
||||
|
||||
## Teardown
|
||||
|
||||
Three layers, stated. Evidence off the box first (R-320): agent journal, controller log and box state,
|
||||
token-leak control 0. **Machine:** VM 334 purged with its disks; `qm list` empty; demo-hp's own containers
|
||||
9201 and 9202 untouched. **Host:** nothing to remove — the drill box WAS the nested VM. **Hub:** the host
|
||||
record deleted, the customer `tester-1` kept with its e-mail, domain and tunnel; **RESET was never used**.
|
||||
Nothing on the off-site server was written, removed or pruned — its listing is empty before and after.
|
||||
Opened: R-539, R-540, R-541, R-542, **R-543** (P1 — off-site on by default is not off-site working on day
|
||||
one), R-544. Closed: R-511, R-534, R-536, R-537, R-538. Register 236 → 244 open.
|
||||
|
||||
## Checks
|
||||
|
||||
`python3 scripts/repo_gates.py --fast` — all 14 gates OK. `scripts/unproven.py --summary` — unchanged at
|
||||
**35 of 55 not walked**. CI for the session's pushes: job **642**, conclusion **success**, matched by
|
||||
`head_sha`.
|
||||
`repo_gates.py --fast` green at every push; controller `go build/vet/test` green; controller gates 15/15;
|
||||
`unproven.py --summary` unchanged at 35 of 55 not walked. Secret-leak check on the evidence: six real
|
||||
secret values as needles, planted control matched 6/6, committed evidence 0.
|
||||
@@ -1,5 +1,41 @@
|
||||
# STATUS — what works, what's broken, what's next
|
||||
|
||||
**Updated 2026-09-16 (evening) — the photos come back, and they open.**
|
||||
|
||||
> **Ready for a volunteer: almost — one thing stands in the way, and it is small.** A brand-new box now
|
||||
> protects the household's own files: I put five photos in, deleted them the way a child would, and got
|
||||
> them back byte-for-byte from the off-site copy. The old route that used to lie now refuses politely
|
||||
> and points at the one that works. What is missing: on a new box the off-site copy is switched on but
|
||||
> **paused** until the household creates their recovery code, and nothing asks them to do it.
|
||||
|
||||
**What changed today.** The backup page stops claiming it holds files it does not hold. A restore that
|
||||
cannot bring your files back now refuses instead of reporting success — and it no longer wipes the app's
|
||||
own wastebasket on the way. „Alkalmazás telepítve" now means installed, not merely started. Every new
|
||||
customer gets the off-site copy by default, 100 GB. The off-site server got the one permission it was
|
||||
missing, so a rebuilt customer's box can be set up again without hand-work.
|
||||
|
||||
**What I proved on a box that installed itself this evening.** It installed from the new image, showed
|
||||
Felhom's own screen with no Proxmox address, registered itself, and **bound with nothing pressed on your
|
||||
side** — the connect e-mail it used was the one the system sent itself. It landed on today's golden.
|
||||
Then: five photos in, the local backup, the recovery-code ceremony, the off-site copy, the deletion, the
|
||||
refusal, the restore, and five photos that open — identical to the originals.
|
||||
|
||||
**Decisions I took.** None under the unattended rule.
|
||||
|
||||
**Needs you.**
|
||||
1. **Say yes or no to publishing the new installer image (1.28.0).** It is built and passed every check,
|
||||
and it fixes the screen that kept showing the pairing code after the box was connected. Nothing is
|
||||
published without your word. If you do nothing: new volunteers keep getting the older image, which
|
||||
works but shows that stale screen.
|
||||
2. **One small fix before a volunteer: tell the household to create their recovery code.** Until they do,
|
||||
the off-site copy is paused — so „your files are protected" is a promise with a delay in it. I can
|
||||
add the prompt and make the sentence state the real state.
|
||||
3. **The slow-crash-loop counter** (yesterday's ruling) is still owed, and is a job for the nightly.
|
||||
|
||||
---
|
||||
|
||||
## Previous note
|
||||
|
||||
**Updated 2026-09-16 (drill on a fresh box) — the fixes hold; the backup promise does not.**
|
||||
|
||||
> **Ready for a volunteer: NO — one reason, and it is new.** On a brand-new box with one drive, the
|
||||
@@ -786,3 +822,4 @@ off. **`peti-felhom` is a real machine we have not heard from since 15 July** an
|
||||
The 2026-08-09 batch (R-279 … R-292), still untriaged; the three remaining R-264 readers; R-317 (one
|
||||
line in the agent); R-327 (decide the naming claim's status); R-359 (nothing reads the off-site store).
|
||||
|
||||
|
||||
@@ -88,7 +88,7 @@ likewise silent. Evidence: `audits/DRILL-r361-2026-08-22/evidence/06-part3-decis
|
||||
|
||||
**NARROWED 2026-08-21 by the backup-truth drill — kept as history; both halves have since closed, see the entry immediately above.** Proven that night on `demo-hp` with planted, hash-recorded files: the **declared-userdata** leg of a **drive-declaring** app does come back byte-identical (`calibre-web`, 5/5 including two Hungarian accented filenames). **Two legs of the same story do NOT:** (a) the off-site restore has **no named-volume leg at all**, so an app's volume tar sits in the unit, in the snapshot and in the checking folder and is never replayed (**R-354**); (b) for the **40 of 53** apps that declare no data drive the off-site restore **refuses outright**, saying a running app „nincs telepítve" (**R-356**). Since the 40-class keeps ALL its data in named volumes, the end-to-end story is **unproven for that class and disproven for the volume leg generally**. The escrow/key half of this row is untouched by that and still stands. Evidence: `audits/DRILL-backup-truth-2026-08-21/evidence/` and `REPORT.md` (2026-08-21). |
|
||||
| **The customer's own UNAIDED recovery journey, end to end** | controller v0.206.0, hub v0.98.0, agent v0.127.0 | **PROVEN-LIVE (2026-08-07, the fifth walk) — an unaided customer journey on a real installation, SCOPED: see what it does not claim** | A throwaway appliance was built from the published ISO on demo-hp, claimed, given three sentinels, escrowed, backed up off-site, then destroyed — guest purged and both enrolled drives wiped. The rebuild and the customer's route were then walked with **no command line inside the guest**. **Data: PASS.** All three sentinels byte-identical (`beb9175d…`, `7c8cb0ad…`, `e012e76f…`), including a 12 MB binary and `C11-őrszem-ékezetes-árvíztűrő.txt` — the Hungarian filename survived disk → restic → SFTP → Storage Box → restore → disk intact. Restored in **16 s** out of the pre-wipe snapshot `f3d9cd67`, non-destructively. **Journey: FAIL.** Four interventions, three of them root on the appliance: R-216 (a correct code called wrong), R-218 (the store never opened; the box stopped asking), R-219 (the promised listing structurally could not render), R-220 (the app could not be redeployed — its drives were unenrollable). Evidence: `tests/campaign11-evidence-2026-08-05/journal.md` | **WHAT IT DOES NOT CLAIM, and what the v0.201.0/v0.97.1 fixes do and do not change.** Six of the nine findings are fixed and each is red-proofed (R-216, R-218, R-219, R-217, R-222, R-215) — but **fixes are not a journey.** This row goes green only when the walk is repeated end to end and completes with no operator intervention. **Three findings are deliberately still open and each is a live blocker for some flow:** **R-214** (the console never stops showing a stale pairing code), **R-220** (after a rebuild the customer's drives cannot be re-enrolled — the deploy refuses and the wizard's list is empty, so no app can be redeployed onto its own data drive), **R-221** (a rebuilt box cannot run the escrow ceremony at all, because the pbsdr convergence marker survives on the host while the seeded `escrow.pbs_storage_id` is rewritten out of `agent.json`). **And R-216's fix does not by itself make a NEW box work:** with the hub guard corrected but the Day-0 manifest still vouching agent 0.120.0, a new box is **HELD, not served** — it stops being lied to, which is better, but the feature does not work for it until agent 0.125.0 is vouched. **⚠ CAMPAIGN-11 PHASES 2 + 4 RAN OVERNIGHT 2026-08-05/06 AND THIS ROW STAYS **FAIL**.** **What they add:** eleven injected faults against the recovery journey, each with a positive control, and a full unattended scheduled cycle. **The backup promise strengthened** — the off-site tier ran itself at 04:15 on a box rebuilt twice and set aside hours earlier (`snapshot_count` 1 → 2), all five daily jobs fired exactly once, and nothing on the must-not list fired. **R-217's and R-215's fixes were proven live under exactly their faults**, the set-aside proved it does not delete (12 535 KB byte-exact at the far end), and a wrong code was refused three times with nothing written and no lockout. **What they do NOT add: any progress on the JOURNEY.** These faults are **not a re-walk** — no customer route was walked end to end — and four new findings say the journey got no better where it matters: **R-224** (a hub outage and a stopped agent are BOTH reported to the customer as a bad recovery code, in 0.056 s and 0.030 s — no unseal attempted; the agent's own `err` distinguishes them and it is discarded at the HTTP boundary, and R-216's gate answers `source=version` so it cannot see reachability — **Phase 1's headline defect relocated from the version channel to the transport**), **R-226** (M1, the only message telling a customer to check their typing, is unreachable on any box that has re-escrowed), **R-225** (an unread store renders `0 pillanatkép · 0 GB` above a card saying it holds backups — 1 snapshot and 12 535 KB were really there), **R-228** (the set-aside history is recorded in `orphaned_renamed_to` and shown by nobody — `OrphanedRenamedTo` has zero references in any template or handler). **§4.1 is now MEASURED rather than deduced** — the floor IS served, from the box's own rendered `GetFloor()` and a cold-started controller's settle-gate line. **§4.2's positive half is still NOT measured** and needs a rebuild. Evidence: `audits/CAMPAIGN-11-recovery-journey-2026-08-05.md`, `tests/campaign11-evidence-2026-08-05/journal-phase24.md` **2026-08-06 — THE FIVE PHASE-2 FINDINGS ARE FIXED AND THIS ROW STILL SAYS FAIL.** controller **v0.202.0** + agent **v0.126.0** close R-224, R-226, R-225, R-227, R-228. **What changed:** the unlock path now classifies WHY it failed, from the value and never the text, and the message that mentions typing is reachable from exactly one class — a real refusal — with **unknown defaulting to neutral**. Proven live on the venue with the same wrong code and only the hub's reachability changLine truncated
|
||||
| **A stranger's FIRST HOUR on a fresh install — download → install → claim → two apps → use → backup → remove → restore, plus a power cut and a mistyped code** | controller **v0.242.0**, agent **v0.130.0**, hub **v0.112.0**, golden **0.242.0**, installer ISO **1.26.1** | **PARTIAL (2026-09-14) — every mechanism PASSED on a fresh box; the UNAIDED journey FAILS before the first app** | `audits/DRILL-fresh-install-0242-2026-09-14.md`, evidence `audits/evidence-drill-fresh-install-0242-2026-09-14/journal.md`. A nested VM on demo-hp installed from the public ISO landed on the vouched set (controller 0.242.0, agent 0.130.0) with no hand upgrade; BookStack and PrivateBin deployed in 68 s / 21 s and were used through their front doors; backup-now moved both dates to the true time; removal with every delete box ticked left no volume, no backup and a 404; a page deleted in BookStack came back **byte-identical with its attachment** 32 s after restore; a hard power cut returned every app on the **same version** with data intact and no alarm; a typo in the code was refused and the right code then accepted, five failures locking for 15 min with an honest Hungarian message. **ONE intervention a volunteer could not make:** the setup-code mail's dashboard link does not resolve for a new customer (no tunnel/DNS is created — **R-494**), so the dashboard was reached by LAN address. **And no instruction exists to begin with (R-493).** **WHAT IT DOES NOT CLAIM:** the self-bind page and the mailed setup code were not exercised (no mailbox in the harness — the operator bind and a box-printed code stood in); DR tier and off-site were off (ep0 fence), so escrow and off-site screens were not walked; no browser, so script-rendered state was not observed. Goes green when a volunteer, from written instructions alone, reaches a working dashboard and a first app with zero interventions. **2026-09-14 (evening) — WALKED AGAIN on ISO 1.27.0/1.27.1 + hub v0.113.0, STILL PARTIAL.** `audits/DOORSTEP-walk-1270-2026-09-14.md`. Held again on a new customer (`tester-1`, real domain + tunnel token): install on three disks and on one, the vouched set, deploy, use, backup-now, removal, byte-identical restore, power cut (same versions), typo + lockout. New and proven: the console is Felhom-only from the FIRST boot on 1.27.1 (VM 332, reboot proven, `pvebanner` masked); the hub tells the operator to hand the passphrase over (live). Still ONE intervention: the tunnel connected but received no routes (R-505), and the record has no e-mail (R-508). The graphical installer was proven only to its password screen (R-507). Goes green on a walk with zero interventions on a published installer. **BIGNIGHT 2026-09-14/15 (ISO 1.27.1):** the first hour re-walked with the real mailbox — self-bind link and mailed setup code both worked; 2 interventions (no auto bind mail R-509; tunnel 502 R-510); then twelve apps, a month of routines and nine faults: `audits/BIGNIGHT-household-month-2026-09-14.md`. **DRILL 2026-09-16 on controller 0.243.0 / agent 0.131.0 / hub 0.115.0 / golden 0.243.0 / ISO 1.27.1 — the WALK half is now PROVEN-LIVE with ZERO interventions, and the BACKUP half is narrowed, not proven.** `audits/DRILL-prove-fixes-0243-2026-09-16.md`. A fresh nested box was installed from the published ISO, landed on this drill's own golden **by checksum** (`e2d1843c…c10a`, no self-update), and was walked as a volunteer: the connect e-mail and self-bind page, the claim, **the tunnel from outside (302→200, R-510 CLOSED)**, the data drive, the file manager with its own generated password (`admin`/`admin` refused 401 before any dashboard action), four apps deployed and used, and „Mentés most” with **26 s** of app downtime. **Interventions: 0** — the two operator presses (self-bind send, PBS re-issue) were pre-declared and counted apart, and the four moments that look like help were my own API-driving errors or my own damage, each named in the drill's interventions table. **The automatic connect e-mail is PROVEN with a real mailbox:** a host delete at 12:22:59Z produced the „Kösd össze a Felhom dobozodat” mail at **12:23:00Z**, with `selfbind_link_sent (host delete)` on the timeline (R-509). **WHAT THIS DOES NOT CLAIM, and it is the backup half of this row's own sentence:** on a one-drive box with no off-site tier — what every fresh install is — the household's files are in **no backup at all** (the whole-guest tiers exclude `mp8` by design; the file leg lives at tier 2/3, both unset), the app-backup page nevertheless reads „DB + Konfig + Adatok” (**R-537**), and a restore reports success while leaving Nextcloud listing five photos that return `Sabre\DAV\Exception\NotFound` — after making the app's own trash, which still held every byte, unreachable (**R-538**). **Also still open from this walk:** the off-site tier cannot be provisioned at all (**R-534**, P1, an ep0 grant), the console keeps its pairing banner after biLine truncated
|
||||
| **A stranger's FIRST HOUR on a fresh install — download → install → claim → two apps → use → backup → remove → restore, plus a power cut and a mistyped code** | controller **v0.242.0**, agent **v0.130.0**, hub **v0.112.0**, golden **0.242.0**, installer ISO **1.26.1** | **PARTIAL (2026-09-14) — every mechanism PASSED on a fresh box; the UNAIDED journey FAILS before the first app** | `audits/DRILL-fresh-install-0242-2026-09-14.md`, evidence `audits/evidence-drill-fresh-install-0242-2026-09-14/journal.md`. A nested VM on demo-hp installed from the public ISO landed on the vouched set (controller 0.242.0, agent 0.130.0) with no hand upgrade; BookStack and PrivateBin deployed in 68 s / 21 s and were used through their front doors; backup-now moved both dates to the true time; removal with every delete box ticked left no volume, no backup and a 404; a page deleted in BookStack came back **byte-identical with its attachment** 32 s after restore; a hard power cut returned every app on the **same version** with data intact and no alarm; a typo in the code was refused and the right code then accepted, five failures locking for 15 min with an honest Hungarian message. **ONE intervention a volunteer could not make:** the setup-code mail's dashboard link does not resolve for a new customer (no tunnel/DNS is created — **R-494**), so the dashboard was reached by LAN address. **And no instruction exists to begin with (R-493).** **WHAT IT DOES NOT CLAIM:** the self-bind page and the mailed setup code were not exercised (no mailbox in the harness — the operator bind and a box-printed code stood in); DR tier and off-site were off (ep0 fence), so escrow and off-site screens were not walked; no browser, so script-rendered state was not observed. Goes green when a volunteer, from written instructions alone, reaches a working dashboard and a first app with zero interventions. **2026-09-14 (evening) — WALKED AGAIN on ISO 1.27.0/1.27.1 + hub v0.113.0, STILL PARTIAL.** `audits/DOORSTEP-walk-1270-2026-09-14.md`. Held again on a new customer (`tester-1`, real domain + tunnel token): install on three disks and on one, the vouched set, deploy, use, backup-now, removal, byte-identical restore, power cut (same versions), typo + lockout. New and proven: the console is Felhom-only from the FIRST boot on 1.27.1 (VM 332, reboot proven, `pvebanner` masked); the hub tells the operator to hand the passphrase over (live). Still ONE intervention: the tunnel connected but received no routes (R-505), and the record has no e-mail (R-508). The graphical installer was proven only to its password screen (R-507). Goes green on a walk with zero interventions on a published installer. **BIGNIGHT 2026-09-14/15 (ISO 1.27.1):** the first hour re-walked with the real mailbox — self-bind link and mailed setup code both worked; 2 interventions (no auto bind mail R-509; tunnel 502 R-510); then twelve apps, a month of routines and nine faults: `audits/BIGNIGHT-household-month-2026-09-14.md`. **DRILL 2026-09-16 on controller 0.243.0 / agent 0.131.0 / hub 0.115.0 / golden 0.243.0 / ISO 1.27.1 — the WALK half is now PROVEN-LIVE with ZERO interventions, and the BACKUP half is narrowed, not proven.** `audits/DRILL-prove-fixes-0243-2026-09-16.md`. A fresh nested box was installed from the published ISO, landed on this drill's own golden **by checksum** (`e2d1843c…c10a`, no self-update), and was walked as a volunteer: the connect e-mail and self-bind page, the claim, **the tunnel from outside (302→200, R-510 CLOSED)**, the data drive, the file manager with its own generated password (`admin`/`admin` refused 401 before any dashboard action), four apps deployed and used, and „Mentés most” with **26 s** of app downtime. **Interventions: 0** — the two operator presses (self-bind send, PBS re-issue) were pre-declared and counted apart, and the four moments that look like help were my own API-driving errors or my own damage, each named in the drill's interventions table. **The automatic connect e-mail is PROVEN with a real mailbox:** a host delete at 12:22:59Z produced the „Kösd össze a Felhom dobozodat” mail at **12:23:00Z**, with `selfbind_link_sent (host delete)` on the timeline (R-509). **WHAT THIS DOES NOT CLAIM, and it is the backup half of this row's own sentence:** on a one-drive box with no off-site tier — what every fresh install is — the household's files are in **no backup at all** (the whole-guest tiers exclude `mp8` by design; the file leg lives at tier 2/3, both unset), the app-backup page nevertheless reads „DB + Konfig + Adatok” (**R-537**), and a restore reports success while leaving Nextcloud listing five photos that return `Sabre\DAV\Exception\NotFound` — after making the app's own trash, which still held every byte, unreachable (**R-538**). **Also still open from this walk:** the off-site tier cannot be provisioned at all (**R-534**, P1, an ep0 grant), the console keeps its pairing banner after biLine truncated
|
||||
| **Off-site app-data capture covers the paths an app declares MANDATORY — on BOTH drive layouts** | controller v0.197.0 | **PROVEN-LIVE (2026-08-04) — and this row was OPTIMISTIC before it** | On demo-hp, an app on the **system-data fallback** bound `/mnt/sys_drive/userdata/media/books` while the capture set looked in `/mnt/sys_drive/felhom-data/userdata/media/books`: the declared-mandatory directory was in **no** snapshot and the run reported `ok` (R-203). After v0.197.0 the capture log reads `1 mandatory path(s)` and the file is **listed inside the snapshot** — `restic ls -l latest --tag calibre-web` → `-rw-r--r-- 1000 1000 181 … /DRILL-SENTINEL.txt`. Evidence: `audits/DRILL-r201-offsite-recovery-2026-08-04.md` §2 and the v0.197.0 CHANGELOG | **WHAT IT DOES NOT CLAIM.** It covers CAPTURE, not RESTORE: **no file has ever been restored from an off-site backup after a wipe** (R-201, ready to resume). It also does not claim the enrolled-drive layout was ever wrong — it was not, which is precisely why this survived. And a run that still misses a mandatory directory now reports **`incomplete`** rather than `ok`, so this row's guarantee is one the status can express. **Widened 2026-08-06 (controller v0.205.0, R-234):** the same verdict now also covers an app skipped ENTIRELY — until then a missing declared FOLDER made the run incomplete while an app with no recovery unit at all still reported `ok`, so the smaller gap moved the verdict and the bigger one did not. **This row still claims CAPTURE, not that a newly-selected app is protected by the next run** — for a DEPLOYED app it is (the run's own pre-dump phase writes the unit, measured 2026-08-06), and for an undeployed one it is not and the card now says so || **Recurring offsite (PBS) whole-guest backups actually LAND, and RESTORE** — local daily + offsite weekly as scheduled work | agent v0.97–0.103, controller v0.174/0.175, hub v0.76.0, host-install 1.20.0 | **PROVEN-LIVE (2026-07-26)** | `audits/SPIKE-r82-phase0-2026-07-26.md`; per-repo CHANGELOGs/REPORTs. **Restore round-trip on demo-hp:** `--selftest=restore-test` against `felhom-pbs:backup/ct/9201/2026-07-26T15:42:42Z` → `pass:true`, `verified:"boot+running"`, **`mount_parity:"ok"`** (`mp0=/var/lib/docker 50G`, `mp1=/mnt/sys_drive 20G`, mp8/mp9 throwaway stand-ins for the archived binds), `source_tier:"pbs"`, 4m5s restore+boot+verify+teardown, scratch band clean afterwards and the live guest untouched. | **This row is distinct from the DR-tier row above, which proves ACTIVATION, not ARRIVAL.** That tier was PROVEN-LIVE as `applied` since 2026-07-21 while demo-felhom held ONE snapshot (a healing artifact) and demo-hp held **zero, ever** — "applied and empty", the R-39 shape one level quieter. **What earns PROVEN-LIVE here:** (a) demo-hp's FIRST EVER offsite backup landed (4.25 GB into a verifiably empty namespace); (b) it **restores into a bootable, mount-complete guest** — `mount_parity` is the non-hollow half, since a boot-only verify cannot see a missing data volume; (c) **the multi-tier quiesce ran through the real UI endpoint** (`POST /api/guest-backup/trigger`, authed+CSRF) and produced **exactly ONE stop/start pair with BOTH backups inside it** — `quiescing 1 stack(s)` 17:01:39 → local done 17:02:56 *"next tier may start (app still quiesced)"* → felhom-pbs snapshotted 17:03:06 → `unquiescing` 17:03:06. **App downtime 1m27s for both tiers**, and the app came back healthy. **Known gaps, recorded not hidden:** the SCHEDULED restore-test still only selects the PRIMARY tier, so the offsite tier is never AUTOMATICALLY restore-tested (the manual/selftest path is proven, the unattended one is not); the hub infers "PBS ⇒ weekly" from storage TYPE rather than a reported cadence; the installer-default fleet flip awaits a full weekly cycle. → **R-82** |
|
||||
| **Restore-proof is UNATTENDED — the scheduler covers EVERY tier, follows the BACKUP rather than the clock, and a failure is heard** | agent v0.104.0 → **v0.121.0**, hub v0.77.0 → **v0.91.0** | **PROVEN-LIVE (2026-08-03)** | per-repo CHANGELOGs; `backlog/SPEC-r85-phase4-5-2026-07-26.md`. Unit red-proofs for tier rotation, restart-survival, the one-heavy-op gate, failure-emits-an-event, and newborn silence. | **The row above is earned by a MANUAL `--selftest=restore-test`; this one is about the SCHEDULED path, and the distinction is the whole point.** Before R-85 the scheduler could only ever see `cfg.Backup.BackupTarget()`, so the offsite tier was never a candidate — and a failed restore-test was a `[WARN]` line with no event at all, which was true for the LOCAL tier that WAS being tested. Now: oldest-first rotation across every configured tier (operator ruling 2026-07-26), persisted so it survives a restart; a restore-test joins the host-wide one-heavy-operation gate so it never contends with a backup over the same tunnel; and the hub raises two DISTINCT operator-tier signals — `restore_test_failed` (broken now) and `restore_test_stale` (unverified, not known-broken), anchored on R-81 so a newborn box never alarms. **Why this was NOT PROVEN-LIVE until now:** rotation had not been observed selecting both tiers across consecutive UNATTENDED cadences — at a 24h cadence a multi-day window — and a single passing run proves the code path, not the schedule. → **R-85**. **R-86 (agent v0.121.0 + hub v0.91.0, 2026-08-03) replaced the schedule and the observation became possible in one afternoon**, because what has to be observed is no longer a multi-day rotation but a RULE: a tier is due when its newest archive that has settled ~24 h has not been proven. **UPGRADE EVIDENCE — a real unattended run on demo-felhom (2026-08-03), triggered by DUE-NESS, not by a timer.** The scheduler's own log: `15:14:38 restore-test tier is DUE … target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-07-28T04:49:43Z … reason="newest settled archive … has not been proven"` → `proxmox-backup-client restore --crypt-mode=encrypt` under the agent's own token → `15:25:08 gate decision class=guest_destroy guest=990000 allowed=true` → `15:25:14 scratch guest torn down` → **`15:25:14 backup: scheduled restore-test passed archive=felhom-pbs:… duration_s=635.1`**. A **14.5 GB encrypted offsite archive pulled from ep0 over the WAN**, restored, booted, verified and destroyed in 635 s, unattended. **The three things a timer could not show**, all verified after it: the state names THAT archive (`{"felhom-pbs":{"archive":"…2026-07-28T04:49:43Z","proven_at":"2026-08-03T13:25:14Z"}}`); a second evaluation reports `due=false … is already proven` and runs nothing; and **an agent restart runs nothing**, which is the defect a person actually noticed — every deploy used to restart the timer. Teardown verified at all three layers: guest absent from `pct list`, **zero** `990000` volumes in `lvs`, and the hub-side `restore_tests[]` entry deliberately RETAINED (it IS the proof the staleness check reads). **R-189 (agent v0.122.0, 2026-08-03) closed the reporting half of this row, and it was a REAL gap in the evidence path:** the proof above reached the hub only because no restart intervened — `restore_tests[]` came solely from an in-memory store, so the 15:25:14 PASS was in fact LOST when the agent restarted 2 m 43 s later for a deploy (`0 restore-tests` on the next two host-reports). Under per-archive due-ness the box would not have repeated the work for a week. The persisted per-tier proof (with the archive, and now the tier) is merged into the report, so a proof survives a restart — one entry per tier, newest wins, and a record that cannot be described honestly is not emitted. **THE HOST TIER'S HALF OF THIS ROW WAS OPTIMISTIC UNTIL 2026-08-03, and it is worth saying plainly.** Every live restore-test cited here is on the OFFSITE tier. The HOST tier was not merely unproven on demo-felhom — it was **unprovable**, and on demo-hp equally: the agent's token had no ACL on `/storage/felhom-backup`, the storage both boxes configure as `local_backup_target`, so the content API answered `{"data":[]}` through the token while root listed three archives. The scheduler skipped it as *"no settled archive yet"* — which is exactly what a brand-new tier reports — so nothing ever said so (**R-185**). **CLOSED 2026-08-03 — agent v0.123.0 + installer 1.24.0.** The grant is applied on both demo boxes (the token now lists 3 and 4 archives), the installer's reuse arm grants on a pre-existing target, and the agent now **asks whether it may read each tier it depends on** instead of inferring it from an empty list: a critical degraded capability naming the storage and the missing role, which the hub alerted and emailed on **while the box was still blind**. **THE HOST TIER IS NOW PROVEN-LIVE, UNATTENDED, ON BOTH DEMO BOXES (2026-08-0Line truncated
|
||||
| Customer RESET (middle lifecycle tier: host delete < RESET < customer Delete): one operator action → pre-first-install; all operational state destroyed, identity + basic config survive | hub v0.61.0, felhom-tenantsync v1.1.0 | **PROVEN-LIVE (external teardown, incl. two real firings)** | **`tests/VALIDATION-n100-rehearsal-2026-07-18.md` — two live firings, both host-delete-first, on two different customers** (`demo-vm-felhom` 15:49:57, `demo-felhom` 16:08:51): every leg `ok` (`claim`, `db_purge`, `descriptor`, `hetzner`, `pbs`), escrow acked separately, each completing in 8–9 s (`hub-state.txt` `customer_resets`). The **Hetzner sub-account destruction is now verified against the live pool box** — and produced the run's sharpest lesson: **a sub-account is an access-control object, not a data object.** Deleting it left its `/home` intact, so re-enabling offsite recreated an account over the previous lifecycle's ciphertext under a key this same RESET had destroyed — which is why the orphan guard fired at 16:58:14 (**a finding by S7's own criterion**) and why RESET now needs a base-dir purge → **R-32**. Prior: hub v0.61.0 REPORT; **ep0 live drill 2026-07-17** (throwaway `drill-reset-01` with a real backup: deprovision `deleted:true` destroyed the namespace + backup group + token, idempotent re-run `deleted:false`, all 3 real tenants + shared user survived); red-proofs (ack-gate, partial-failure resumability) + orchestration/store/offsite/render tests | External teardown FIRST, DB purge LAST, every leg idempotent; refuses while any host row exists; separate escrow-custody ack; clears claim (fresh code next onboarding); keeps the offsite tier CHOICE, drops provisioned fields. **Live-clicked 2026-07-18** (twice, by Viktor) — this supersedes the earlier "not live-clicked / Hetzner delete unit-tested only" note. **R-25b CLOSED (hub v0.69.0, 2026-07-21):** the Danger-zone DELETE is now the guided full-teardown cascade that runs this very sequence as its middle leg — see the row below |
|
||||
|
||||
@@ -20,3 +20,64 @@ VMID Status Lock Name
|
||||
## against all 41 evidence files. Planted positive control matched 6/6, so the grep works; the
|
||||
## committed evidence matched 0 files. The earlier count of „22" was the WORD „password" in labels
|
||||
## like „Password set" — a word count, not a leak check.
|
||||
## 2026-09-16T18:16:43Z TEARDOWN, LAYER 3 — delete the HOST record, keep the customer (never RESET)
|
||||
pre-state: {"deletable":true,"escrow_present":true,"guests":1,"log_bundles":0,"pbs_secret_present":true,"recovery_present":true,"reports":9,"status":"stale","wg_peer_bound":true}
|
||||
delete POST at 2026-09-16T18:16:43Z
|
||||
http=409
|
||||
host record after: 200 (404 = gone)
|
||||
customer record after: 200 (200 = KEPT)
|
||||
hub log right after the delete:
|
||||
2026/09/16 20:16:30 [INFO] Host staleness: tester-1-33b6a9 ok → stale (host_stale)
|
||||
2026/09/16 20:16:31 [INFO] Operator email sent for tester-1/host_stale
|
||||
2026/09/16 20:16:43 [WARN] host delete refused: tester-1-33b6a9 has key escrow (acknowledgement missing)
|
||||
## 2026-09-16T18:17:45Z delete retried WITH the escrow acknowledgement (retained custody)
|
||||
delete POST at 2026-09-16T18:17:45Z
|
||||
http=303
|
||||
host record after: 404 (404 = gone)
|
||||
customer record after: 200 (200 = KEPT)
|
||||
hub log right after:
|
||||
2026/09/16 20:16:31 [INFO] Operator email sent for tester-1/host_stale
|
||||
2026/09/16 20:16:43 [WARN] host delete refused: tester-1-33b6a9 has key escrow (acknowledgement missing)
|
||||
2026/09/16 20:17:30 [INFO] Staleness: tester-1 ok → stale (node_stale)
|
||||
2026/09/16 20:17:31 [INFO] Operator email sent for tester-1/node_stale
|
||||
2026/09/16 20:17:46 [INFO] host deleted: tester-1-33b6a9 (escrow deleted: true)
|
||||
2026/09/16 20:17:46 [INFO] self-bind link emailed to the registered address of tester-1
|
||||
2026/09/16 20:17:46 [INFO] self-bind link (hash 69b8422e…, valid 7 days) emailed to the registered address of tester-1
|
||||
2026/09/16 20:17:46 [INFO] self-bind link auto-minted for tester-1 on host delete (the console banner's promised email now exists)
|
||||
## THE FIRST DELETE WAS REFUSED, and it was right to refuse (18:16:43Z):
|
||||
## „host delete refused: tester-1-33b6a9 has key escrow (acknowledgement missing)" -> HTTP 409,
|
||||
## host record still 200, customer still 200 — nothing was dropped.
|
||||
## This box carries a REAL key escrow because the household performed the ceremony an hour ago, and
|
||||
## the hub will not discard a customer's sealed package on an unacknowledged delete. The documented
|
||||
## action is to acknowledge it, which moves the escrow to RETAINED custody rather than destroying it.
|
||||
## AND THE STALENESS ALARM FIRED TRUTHFULLY: „Host staleness: tester-1-33b6a9 ok → stale (host_stale)"
|
||||
## at 20:16:30 CEST with an operator mail one second later — the box really was gone by then.
|
||||
## LAYER 3 DONE (18:17:45Z), with the acknowledgement given:
|
||||
## POST /hosts/tester-1-33b6a9/delete (confirm_host_id + delete_escrow=1) -> 303
|
||||
## host record -> 404 (gone) customer record -> 200 (KEPT; RESET was never used)
|
||||
## hub log, same second:
|
||||
## „host deleted: tester-1-33b6a9 (escrow deleted: true)"
|
||||
## „self-bind link emailed to the registered address of tester-1"
|
||||
## „self-bind link (hash 69b8422e…, valid 7 days) emailed to the registered address of tester-1"
|
||||
## „self-bind link auto-minted for tester-1 on host delete (the console banner's promised email
|
||||
## now exists)"
|
||||
## So R-509's automatic e-mail fired again, on a second box, at the second of the delete.
|
||||
## A WORDING MISMATCH WORTH CHECKING RATHER THAN REPEATING: the refusal text says the acknowledgement
|
||||
## moves the escrow „to retained custody", while the log line says „escrow deleted: true". Those are
|
||||
## two different statements about a household's last key, so the customer record is read below
|
||||
## instead of trusting either sentence.
|
||||
## THE AUTOMATIC CONNECT E-MAIL — PROVEN AGAIN, on a second box in the same session:
|
||||
## host delete at 2026-09-16T18:17:45Z -> mail in the customer's inbox at 18:17:46Z (ONE second),
|
||||
## „[Felhom] Kösd össze a Felhom dobozodat", to tester1@felhom.eu, body „Elkészült a Felhom dobozod,
|
||||
## és készen áll az összekötésre… https://hub.felhom.eu/bind/…", link valid 7 days.
|
||||
## Requirement was two minutes. R-509 now has two independent live proofs today (12:23:00Z and
|
||||
## 18:17:46Z), on two different boxes.
|
||||
## THE ESCROW'S FATE, answered from the hub's own text rather than from either log line:
|
||||
## „host deletion only demotes custody, never destroys it"
|
||||
## „1. The host(s) will be deleted — recovery-key custody is demoted to RETAINED custody, not
|
||||
## destroyed." …and the customer delete is named as „the one true purge point".
|
||||
## So the acknowledgement demoted the custody; it did not destroy the household's sealed package.
|
||||
## The log line „escrow deleted: true" is the FLAG's name, not the effect — recorded as a row.
|
||||
## CUSTOMER RECORD AFTER EVERYTHING: present (200), with e-mail, domain, tunnel and DR tier intact,
|
||||
## and „waiting — no host enrolled yet — the Day-0 install enrolls…" — exactly the state a customer
|
||||
## is in between boxes. RESET was never used.
|
||||
@@ -726,6 +726,7 @@ class (an image `VOLUME` at an unmounted path) is still live — `immich-server`
|
||||
| **R-541** | **[P3-LOW] There is no path to move a customer between off-site boxes, or from shared to dedicated.** Read from source 2026-09-16: provisioning is idempotent-reuse keyed on the customer (`shared already provisioned for tester-1 (subaccount 311327)`), and a dedicated deprovision destroys the repository — so "move this customer" has no safe route today. It becomes reachable the moment R-540's second pool box exists, or when a customer outgrows the shared model. **Needs:** a move that copies the repository, re-keys, and only then releases the old sub-account — a new mechanism nobody has measured. | **READY — rank P3-LOW; owner: CC (hub) — design first** |
|
||||
| **R-542** | **[P3-LOW] `/api/disks/candidates` offers a REGISTERED, in-use drive under „initialize".** MEASURED 2026-09-16 on the fresh box (controller 0.244.0): after `/dev/sdb` was formatted, mounted at `/mnt/felhom-drives/adatlemez` and registered as the default data drive, the endpoint still listed it under `initialize` (and again under `attach` with `already_mounted: null`). **Customer-invisible today:** the Meghajtók page filters correctly — its „Nem regisztrált meghajtók" section lists none, and the drive shows as „Adatlemez · Alapértelmezett · Aktív". So the defect is in the raw endpoint that feeds a FORMATTING flow, not in the page. **It also misleads a session:** I read „not mounted" off this endpoint and briefly filed a false finding against the product (corrected in `audits/evidence-backup-promise-2026-09-16/phaseE-freshbox.txt`). **Fix shape:** exclude paths the controller has registered from `initialize`, and set `already_mounted` from the real mount state rather than null. | **READY — rank P3-LOW; owner: CC (controller/agent)** |
|
||||
| **R-543** | **[P1-HIGH] Off-site ON by default is not off-site WORKING: on a fresh box tier 3 sits at „Kulcsletétre vár" until the household does the escrow ceremony, and nothing asks them to — while the tier-1 row now tells them their files are protected by that very copy.** MEASURED 2026-09-16 on the fresh box (controller 0.244.0, hub 0.116.0, off-site provisioned automatically by the new default): the app-backup page reads „3. mentés — Kulcsletétre vár · A távoli mentés a titkosítási kulcs letétbe helyezéséig szünetel", the remote page reads „Helyreállítási kód szükséges", and `POST /backup/offbox/run` returns 302 while producing no snapshot (the controller log shows only `offsite-credential-retry`, no restic activity). **Why it matters more than before today:** hub v0.116.0 makes off-site the default *because* a one-drive box otherwise keeps the household's files in no tier at all (R-537/R-538), and controller v0.244.0 now prints „Az alkalmazás fájljait a távoli másolat (és a második meghajtó) védi" under the tier-1 row. On day one both are true-in-intent and false-in-fact: the copy is paused. **Fix shape (one of):** prompt the escrow ceremony as part of first-run when off-site is enabled and un-escrowed; and/or make the tier-1 sentence state the tier's actual state („…védené — a távoli mentés a helyreállítási kód létrehozásáig szünetel"). The ceremony itself works and is customer-facing („Helyreállítási kód létrehozása"); what is missing is that anyone is told to do it. | **READY — rank P1-HIGH; owner: CC (controller copy + first-run prompt)** |
|
||||
| **R-544** | **[P3-LOW] The host-delete log line says „escrow deleted: true" while the documented (and actual) effect is DEMOTION to retained custody.** MEASURED 2026-09-16 during the teardown of the fresh box: an unacknowledged delete was correctly refused 409 („has key escrow (acknowledgement missing)") and the refusal text promises the acknowledgement „moves it to retained custody"; the acknowledged delete then logged `host deleted: tester-1-33b6a9 (escrow deleted: true)`. The hub's own customer page states the truth — „host deletion only demotes custody, never destroys it… recovery-key custody is demoted to retained custody, not destroyed", with the customer delete named as „the one true purge point". **Nothing is broken; the log is.** An operator reading that line during an incident would believe a household's last key had just been destroyed, and the R-304 retention exists precisely so it is not. **Fix shape:** log what happened — `escrow custody demoted to retained (host delete)` — and keep the boolean's name out of operator-facing text. | **READY — rank P3-LOW; owner: CC (hub)** |
|
||||
| **R-537** | **[P1-HIGH] The app-backup page labels the tier-1 backup „DB + Konfig + Adatok" and prints the app's data-drive size next to it — but the tier-1 unit contains NO drive-side app data at all.** MEASURED 2026-09-16 on the drill box (fresh install, controller 0.243.0, one drive, tier 2 and tier 3 both „Nincs beállítva"): five photos (3 000 000 B) were uploaded into Nextcloud through its own WebDAV interface, then the customer-visible „Mentés most" was pressed (`POST /api/backup/run` → 200, the unit grew 25 337 B → 978 MB). The resulting unit's `manifest.json` lists `db-dumps` + three **docker volume** dumps and nothing else; listing the 781 MB `nextcloud_nextcloud_html.tar` (29 346 entries, positive control `version.php` = 3 hits) gives **`Fotok` = 0 and `nyaralas` = 0**, and `./data/` is the empty bind-mount point. A `find` over the whole `backups/` tree for `*appdata*` / `*Fotok*` returns nothing. The page nevertheless renders „1. mentés … DB + Konfig + Adatok" and „Nextcloud Adatlemez 65.1 MB" — a size measured on exactly the data it does not copy (`internal/web/handlers.go:1176-1178`, `BackupContents`). **This is a truth defect, not a design defect:** `07-backup-architecture.md` §6.2 places nextcloud's file leg at **Tier 2 and Tier 3 only**, and its „[FACT] What the whole-guest tiers do NOT carry" says `mp8 /mnt/felhom-drives` is out of vzdump scope (confirmed live: „excluding bind mount point mp8 … (not a volume)"). So on a one-drive box with no off-site tier — the state every fresh install starts in — the household's files are in **no backup**, while the page says „Adatok". Same family as R-517/R-518. **Fix shape:** render tier-1 contents from the capture set actually written (`ComputeCaptureSet`), so a unit with no file leg reads „DB + Konfig" and the drive size is not shown beside it; and say on the page that the app's files need tier 2 or tier 3. Evidence: `audits/evidence-drill-0243-2026-09-16/phase2-f10.txt`. **CLOSED 2026-09-16 — controller v0.244.0, proven live.** The contents label is computed PER TIER from what that tier captures: Tier 1 says „Adatok" only when the app's data really is in the volumes the unit captured, and a class-A app carries one sentence saying where its files ARE protected. Proven on demo-hp through the page the customer opens: Paperless-ngx reads „1. mentés … DB + Konfig" with „Az alkalmazás fájljait a távoli másolat (és a második meghajtó) védi …", while its „2. mentés" row still reads „DB + Konfig + Adatok". Red-proof: restoring the old app-shaped label fails `TestAppBackupRows_Tier1LabelDoesNotClaimFilesItCannotHold`. **RE-PROVEN 2026-09-16 on a FRESH box** (installed from the built ISO 1.28.0, controller 0.244.0, off-site on by default): the Nextcloud row read „1. mentés … DB + Konfig" with the new sentence, „2. mentés … Nincs 2. (off-drive) másolat", „3. mentés Sikeres restic → …your-storagebox.de"; „DB + Konfig + Adatok" appeared ZERO times while the local unit held no file leg. | **CLOSED 2026-09-16 — controller v0.244.0 (proven live on demo-hp)** |
|
||||
| **R-538** | **[P1-HIGH] A tier-1 app restore reports plain success and leaves Nextcloud listing files whose bytes were never in the backup — and it destroys the app's own trash, the customer's last copy.** MEASURED 2026-09-16 on the drill box, F10 („a child deletes the photo folder"): the five photos were deleted through Nextcloud (DELETE 204, PROPFIND 404), then restored through the page exactly as a customer would (`POST /backup/restore` `stack_name=nextcloud` `snapshot_id=helyi` → 302, finished in **35 s**, „A(z) nextcloud: 3 adatkötet és az adatbázis visszaállítva — az alkalmazás újraindult."). Afterwards the folder is back and **lists all five photos**, and **none of them opens**: `GET nyaralas-1..5` = 404 / 503×4 with `Sabre\DAV\Exception\NotFound`, while the positive controls at the same moment pass (`status.php` 200, WebDAV PUT 201, GET 200). Cause: the replayed MariaDB dump (11:01:45Z) knows the photos, the bytes live on `mp8` and were never captured (R-537). **Worse:** the bytes were still on the drive in Nextcloud's own trash (`appdata/nextcloud/admin/files_trashbin/files/Fotok.d1789556707/nyaralas-1..5.jpg`, all five present) and the restored database no longer references them — the trash listing comes back **empty**, so „restore from trash", the one route that would have worked, is gone. The customer is left with five unopenable photos, a success message, and no warning. **Fix shape:** before replaying a database whose app has an uncaptured file leg, refuse or warn („ennek az alkalmazásnak a fájljai nincsenek ebben a mentésben — a visszaállítás után a fájlok hiányozni fognak"); and never present a DB-only restore of a class-A app as a complete one. Evidence: `audits/evidence-drill-0243-2026-09-16/phase2-f10.txt`. **CLOSED 2026-09-16 — controller v0.244.0, proven live.** A unit restore refuses before anything is touched when the unit cannot return the app's drive-side files, and names the route that can. Fired live on demo-hp: `POST /backup/restore` for paperless-ngx → 302 with „Ez a mentés nem tartalmazza az alkalmazás fájljait, ezért nem állítjuk vissza az adatbázist föléjük — a fájlok így a helyükön maradnak. A fájlok a távoli másolatból állíthatók vissza …", and the app read `running` before AND after, so nothing was stopped and no trash was made unreachable. The database-and-settings-only path exists as a separately worded second step. Red-proof: disabling the guard fails `TestUnitRestore_RefusesWhenTheUnitCannotHoldTheFiles`. **RE-PROVEN 2026-09-16 on a FRESH box, and this time the refusal had somewhere to point:** after five photos were deleted, `POST /backup/restore` was refused with „…a fájlok így a helyükön maradnak. A fájlok a távoli másolatból állíthatók vissza: … „Teljes visszaállítás (fájlok + adatbázis)"", the app read `running` before AND after, and the wastebasket was untouched. The off-site route then returned all five photos — 200 with the exact uploaded sizes and sha256 IDENTICAL to the originals, 5/5, with a negative control. Evidence: `audits/evidence-backup-promise-2026-09-16/phaseE-photos.txt`. | **CLOSED 2026-09-16 — controller v0.244.0 (proven live on demo-hp)** |
|
||||
| **R-525** | **[P3-LOW] FileBrowser has its own login; putting it behind the dashboard session (traefik forwardAuth or Quantum proxy auth) is a new mechanism nobody has measured.** Filed 2026-09-15 by the P1-fixes task (B.5). R-513 closed the default-password hole with a generated password; a household still has two logins. **What it needs:** a spike on a scratch guest — forwardAuth to the controller session, and what FileBrowser Quantum does with a trusted header. | **READY — rank P3-LOW; owner: CC (spike)** |
|
||||
|
||||
Reference in new issue
Block a user