diff --git a/documentation/architecture/00-capability-map.md b/documentation/architecture/00-capability-map.md index b97286b1..5442c8dd 100644 --- a/documentation/architecture/00-capability-map.md +++ b/documentation/architecture/00-capability-map.md @@ -181,6 +181,7 @@ | [D13](capability-map-detail/D.md#d13) | Media apps see EVERY drive; a drive added, pulled, back or safely removed is handled per app (notice, real restart, wiring, torrents paused/resumed) | controller v0.313.0, catalog `0188430`, hub v0.151.0 | **PROVEN-LIVE** | Tier-0 boxes (Tester 1, demo-hp 9201); the fleet with golden 0.313.0 + the floor | `audits/media-build-2026-10-15/G/` (2026-10-15); `audits/media-drive-events-2026-10-14/` (2026-10-14) | | [D14](capability-map-detail/D.md#d14) | A second local copy COUNTS only on a proven different physical disk: the agent knows each place's disks; a VM box makes the copy but says it cannot check; a same-disk copy is not made; a NAS never hides the one-copy notice; a box with no drive refuses a data-folder app in words | agent v0.157.0, controller v0.317.0, hub v0.154.0 | **PARTIAL** | live: VM `unknown` (Tester 1) and two real-disk boxes `proven` (demo-hp, N100); unit only: the same-disk skip, the NAS notice, the no-drive sentence; `usb_bridge` never seen on a device | `audits/storage-identity-buildA-2026-10-15/README.md` (2026-10-15); `architecture/16-storage-physical-identity.md` §13 | | [D15](capability-map-detail/D.md#d15) | A one-disk box gets its own data area („Belső tárhely") outside every whole-box copy, registered as the default drive with no press, so media and data-folder apps install with no extra step; a free disk's partitions can be formatted or used one by one | installer 1.33.0, agent v0.158.1, controller v0.318.0, golden 0.318.0 | **PROVEN-LIVE** | fresh 512 GB one-disk VM on demo-hp; partitions on a scratch disk; the old-box signed job and the D4 sentence unit-proven only; an empty part asks for confirmation (R-1093) | `audits/storage-identity-buildB-2026-10-15/README.md` (2026-10-16) | +| [D16](capability-map-detail/D.md#d16) | A failed or cut restore puts the household's data back EXACTLY as it was and starts the app — local unit, second drive, kept load and off-site, and after a power cut or a killed controller (D22) | controller `main` (0.338.0-rc.1), unreleased | **IMPLEMENTED** | proven live on 9202 (rc.1): a damaged unit and a controller killed between the volume step and the replay; the second-drive pre-check (R-1178), the mirror skip (R-1180) and the off-site power cut by test only; a database outside the replaced volumes is not covered | `audits/night-2026-10-21/P3-restore/02-run.log` (2026-10-21) | ## E. Access, networking & household use diff --git a/documentation/architecture/03-host-agent.md b/documentation/architecture/03-host-agent.md index 3103936a..a93a4b4e 100644 --- a/documentation/architecture/03-host-agent.md +++ b/documentation/architecture/03-host-agent.md @@ -655,6 +655,20 @@ run as a **journaled reconcile job**; a mid-flight failure is compensating-rolle the just-restored guest — allowed unsigned per §4, same-transaction provenance). They diverge in the *archive* and the *back half*, **and in identity policy** (below). +**Guest swap = guest RAM, and the host's swap behind it (R-1192; agent `main`, unreleased — the release is D26).** +`[FACT]` 2026-10-21 (`audits/fixes-2026-10-21/P4-freeze/`): Proxmox gives an LXC a 512 MB swap cap when nothing sets +`swap`, and the agent never set it; on the 8 GB minimum machine (guest 4096 MB) the household's first uploads filled the +guest with its swap full, the kernel could evict only program code, and the box froze (memory „full" pressure 96 %, load +162, the dashboard dead 6½ minutes). `[DESIGN]` The size step sets `swap` = the guest's memory: at bring-up, at a RAM +resize (`POST /guest/memory`), and once a minute after agent start for every pool LXC whose swap is below its RAM (live, +no restart, never lowers). Proven on a fresh box the night of 2026-10-21 (`audits/night-2026-10-21/P1-soak/`: 512 → +4096 one minute after the fixture agent started; the 2-hour household load then answered every 10-second probe). **The +host must have swap behind it:** our installer gives an 8 GB / 256 GB machine 7.75 GB (measured the same night); a BYO +host may have none. The agent reports `host.host_swap_mb` and `host.guest_swap_mb` on every host report, logs one WARN +when the first is below the second, and the hub mails the operator once (`host_swap_short`, operator-only); the +controller's install page adds „A gépen nincs elég cserehely (swap)…" when memory is tight and the agent says so. The +agent never creates or enlarges host swap — changing a host its owner set up is a decision (register). + **Identity reset is scenario-specific — this is a correctness boundary, not a detail.** "Reset identity" is shorthand for two different operations: diff --git a/documentation/architecture/07-backup-architecture.md b/documentation/architecture/07-backup-architecture.md index cf293239..ac9bcc99 100644 --- a/documentation/architecture/07-backup-architecture.md +++ b/documentation/architecture/07-backup-architecture.md @@ -750,6 +750,29 @@ forced failure as on 2026-10-19 → 36 tables, the backup-only table absent, sch the app running, no hold, the copy (69 MB, 0.9 s) gone; a normal restore of the same snapshot then worked. The R-893 known limit above is closed for named volumes; placed FILES are still not taken back (the plain-files mix, as ruled). +**`[DESIGN]` 2026-10-21 — the put-back covers EVERY restore, and a power cut (operator ruling D22 = yes, 2026-10-21 +20:05; controller `main`, unreleased).** (1) **The local unit restore** — the app's own unit, the second drive's unit, +the second drive's „Teljes visszaállítás" and a kept load all run `RestoreFromRecoveryUnitAtWith` — now takes the same +measure → copy → put-back (`offbox_restore_undo.go`, one implementation, no second copy of the logic): the copy must fit +beside the unit before anything stops; ANY failure after the stop (volume replay, definition write, DB-only start, DB +replay, start) pours the live volumes back, writes the live definition back and starts the app; a put-back that fails +holds and keeps the copy; with no copy (a unit with no volume archive) a half-done DB replay HOLDS (the R-379 ruling) +instead of starting on it. The window carries the app-stop marker (`unit_restore`). (2) **A restore cut by a power loss +or a controller kill** (R-1195): once the copy is whole the restore writes `undo.json` and the live definition files +beside it and records the folder in the app-stop marker; at start the crash recovery pours it back BEFORE it restarts the +app, and the household reads „A visszaállítás félbeszakadt (áramszünet). Az alkalmazás adatai a visszaállítás előtti +állapotban vannak." A cut before the copy was whole restarts as before (nothing had moved). Local and off-site alike. +(3) **The second drive's „Teljes visszaállítás" asks the unit's refusals before its file half** (R-1178), so a refusal +moves nothing and its „érintetlenek" is true; a later refusal starts the app and names the changed files. (4) **The +night's second-drive mirror skips an app while a restore of it runs** (R-1180). **`[FACT]` 2026-10-21 night (9202, +controller 0.338.0-rc.1, `audits/night-2026-10-21/P3-restore/`):** Rallly + Postgres; a damaged unit volume → the live +volume poured back, the newest row still there (A,C), the app running, „…Nem változott semmi…" (on 0.331.0 the same case +crash-looped Postgres and lost the row, C3 2026-10-20); the controller killed (kill -9) right after the volume step → +on start „interrupted restore is put back … 1 volume(s) poured back, definition=true", markers A,C,D (D = the row newer +than the unit), the app running, the record reads the D22 sentence. Known limit kept: a database whose data is NOT in a +named volume the unit replaces (none in the catalog today — a drive-file app's unit restore is refused by R-538) is not +covered by the volume copy. + **[DESIGN] 2026-08-22 — the failure ladder of a database restore: replay → rollback → hold.** Recorded here rather than only in a closed register row, because a decision that survives only inside a closed work item is a decision nobody will find. diff --git a/documentation/architecture/capability-map-detail/D.md b/documentation/architecture/capability-map-detail/D.md index 6ede0e7b..12fa8b7c 100644 --- a/documentation/architecture/capability-map-detail/D.md +++ b/documentation/architecture/capability-map-detail/D.md @@ -294,3 +294,29 @@ part next to a mounted part can only be used, not formatted (the root wrapper tr --- +## D16 — A failed or cut restore puts the data back exactly (D22) + +**What it is:** a restore that fails after it began changing an app — or is cut by a power loss or a killed controller +— leaves the app on the household's own data as it was before the restore, running, and says so. Before (2026-10-20, +C3 on 9202): a damaged local unit removed the live database volume, Postgres crash-looped, the newest row was gone and +nothing could put it back; a power cut mid off-site restore left the app on the backup's data (rehearsal s2). + +**Components:** controller `main` (`d582feb`, unreleased): `restore_unit.go` reuses `offbox_restore_undo.go` (measure, +fit, copy, put back, hold on a double failure); `restore_crash_putback.go` + the app-stop marker's `undo` record and +`Recover`'s put-back; `tier2_whole.go` pre-check (R-1178); `RunAllTier2` skip during a restore (R-1180). Design: `07` §6.3 +(the 2026-10-21 `[DESIGN]` paragraph), operator ruling D22. + +**Status:** **IMPLEMENTED** — proven live on scratch 9202 with 0.338.0-rc.1 (2026-10-21): a damaged unit volume → the +live volume poured back, the newest row kept (A,C), the app running, „…Nem változott semmi…"; the controller killed +(kill -9) right after the volume step → on start the copy poured back (A,C,D — D newer than the unit), the app running, +„A visszaállítás félbeszakadt (áramszünet). Az alkalmazás adatai a visszaállítás előtti állapotban vannak." A normal +restore is unchanged (the control). PROVEN-LIVE when the release reaches the fleet. + +**Unit only:** the off-site power cut (`TestR1195_AKilledOffsiteRestoreIsPutBackOnStart`), the second drive's pre-check +(`TestR1178_*`), the mirror skip (`TestR1180_*`), a failed put-back at start (`TestR1195_AFailedPutBackOnStart…`). +**Known gap:** a database whose data is not in a named volume the unit replaces is not covered by the copy (none in the +catalog today; a drive-file app's unit restore is refused by R-538). + +**Evidence:** `audits/night-2026-10-21/P3-restore/` (the run log, the red-proofs). + +--- diff --git a/documentation/audits/night-2026-10-21/P1-soak/00-provenance.txt b/documentation/audits/night-2026-10-21/P1-soak/00-provenance.txt new file mode 100644 index 00000000..d2763126 --- /dev/null +++ b/documentation/audits/night-2026-10-21/P1-soak/00-provenance.txt @@ -0,0 +1,16 @@ +Soak box provenance (night 2026-10-21, Part 1) +- VM 9322 `night1021-soak` on demo-hp (4 vCPU, 8 GiB, 256 GB raw system disk + 200 GB raw data disk on nvme-scratch), + created 18:19:30 UTC; installed from the public ISO felhom-installer-1.29.0-pve9.2-1.iso (Install pressed 18:23:34). +- Customer Tester4 (the kept fresh-install customer; its old host Tester4-f6b67c was deleted by the operator 16:49 UTC). + A NEW customer was not created: creating one provisions on ep0 and the Storage Box (read-only tonight). +- Host enrolled as Tester4-9aca27 (hub 20:33:27 CEST); guest 9201 (tester4), 192.168.0.125; host 192.168.0.128. +- Golden served by the hub manifest: 0.331.0; the box self-updated to controller 0.337.0 (hub log 20:35:52 CEST). +- Agent as installed: 0.161.0 (sha256 ff4e6dc5…). Fixture 1 (18:36:13 UTC, by hand): 0.162.0-night.1 = felhom-agent + 286a7e3 (the swap fix), sha256 f9fa22dc…; guest swap raised 512 → 4096 at 18:37:13 UTC by the start-up check. + Fixture 2 (20:51:20 UTC, by hand): 0.162.0-night.2 = felhom-agent 0da792e (+ host-swap report, R-1194), sha256 d9c2f8b6…. + The release candidate built from the same commit with version 0.162.0: sha256 5fd7a23b…. +- Load: tools/soakrun.sh 120 on helper CT 9332 (night1021-loader, demo-hp): load.py (the rehearsal's first uploads) at + 18:52:32 UTC, then load2.py cycles; generator seed 20261020 (gen_household.py), 3414 files, manifest equal to the + rehearsal's A5 manifest except 4 generated PDFs (PDF timestamps). +- Recorder: tools/rec.py on the host (10 s, VMID 9201); outside probe tools/outside.sh on DooPlex (10 s /api/health, + QEMU disk counters); from 20:4x UTC the probe writes to the scratchpad and is copied back at the end. diff --git a/documentation/audits/night-2026-10-21/P1-soak/04-agent-fixture.txt b/documentation/audits/night-2026-10-21/P1-soak/04-agent-fixture.txt index c81dc11a..f74d6c33 100644 --- a/documentation/audits/night-2026-10-21/P1-soak/04-agent-fixture.txt +++ b/documentation/audits/night-2026-10-21/P1-soak/04-agent-fixture.txt @@ -16,3 +16,9 @@ memory: 4096 swap: 4096 4294967296 18:37:18 +swap2 start 20:51:20 +d9c2f8b65f2573c2da8db9aa5c1fcc78a53de7a831ec7ea1afb8ea2e26c18e81 /usr/local/bin/felhom-agent +active +felhom-agent 0.162.0-night.2 +memory: 4096 +swap: 4096