From 99c0f5e3107c8519d8e640c71c729c71d04288b6 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Mon, 26 Oct 2026 20:11:22 +0100 Subject: [PATCH] =?UTF-8?q?D55=20=3D=20260=20recorded=20(09=20=C2=A73=20de?= =?UTF-8?q?cision=20268);=20the=20grant=20set=20and=20read=20back;=20R-129?= =?UTF-8?q?9=20filed=20(no=20grant=20button)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- CONTEXT.md | 6 ++++++ STATUS.md | 5 ++--- documentation/architecture/09-update-architecture.md | 9 +++++++++ documentation/backlog/OPEN-ITEMS.md | 3 ++- documentation/backlog/STATUS-HISTORY.md | 8 ++++++++ 5 files changed, 27 insertions(+), 4 deletions(-) diff --git a/CONTEXT.md b/CONTEXT.md index a2fd026b..0afb63a4 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -16,6 +16,12 @@ > and holds nothing of its own; this file does hold its own content, namely the standing rulings below. +> **2026-10-26 (night) — D55 = 260 (`09` §3 decision 268); the grant is SET.** CC used the operator's one-purpose token +> `HUB_TOKEN_D55_WINDOW` („cc-d55-window", Off-site write only) once: `POST /offsite/window-grant/demo-hp` `max_remove=260` +> → 200, 19:10:31Z; read back `offsite_window_grant:demo-hp = max:260` (hub DB with WAL) and the hub log line „granted ONE +> clean-up window … 260". The grant is consumed at demo-hp's next window request (after its next nightly off-site run). +> The token is the operator's to revoke after that. The hub has no page button for the grant (R-1299). + > **2026-10-26 (evening run) — R-1297 A built (`ecd82aba`, `e33ec821`); D53 built, commit only (controller `5bd03ad`, hub > `7e57e824`); D54 blocked on a scope.** CI: `script_tests_gate.py` `PATH_FILTERED` (decoys / due-checks / dooplex-offsite > run on a branch push only when their paths change; `FELHOM_CHANGED_FILES` set only for `push:refs/heads/*` in diff --git a/STATUS.md b/STATUS.md index 5976f178..bfffaaa6 100644 --- a/STATUS.md +++ b/STATUS.md @@ -2,7 +2,7 @@ **Ready for Tester 2? Yes — tell the friend go (hand-over list: `documentation/pilot/TESTER2-handover.md`).** -**Updated 2026-10-26 (evening).** CI is faster: two runners work at once, and a hub or document push now takes about 2 minutes instead of 6–13. Held apps now reach you (built, not released): a „Held apps" list on the host page and the Hosts page, and one mail after a waiting time. One pick for you: D55. +**Updated 2026-10-26 (night).** Your D55 token worked: demo-hp gets one clean-up window with a cap of 260 at its next nightly off-site run. Tonight CC fixes data-safety holes (saved only); Tuesday: the line test, then each release on your yes. ## What works - **The Plex attack (10-17 → 10-23) did not reach Felhom's secrets** (R-1241). @@ -32,7 +32,6 @@ Newest first. CC's pick is in the „Pick" column. Each row also has its full te | # | Row | Question | Pick (why) | The other option costs | If you do nothing | Blocked | |---|---|---|---|---|---|---| -| D55 | R-1294 | CC's hub token may not grant demo-hp's one clean-up (the hub says „missing scope offsite:write"), so you grant it: hub → Off-site → demo-hp → one window with a raised cap. Which cap? | **260** (the old-copy list grows about 14 a night: 209 on Sunday, about 237 by Wednesday night's window; the box still removes only what its normal rule removes, and the fake-copy guard still runs) | 220 as first ruled: it is refused again, nothing is deleted, one error mail | demo-hp stays stuck and mails an error every Sunday | — | | D38 | R-1235 | Put the newest golden (637 MB) into the nightly release copy on ep0 too? | **Yes** (a new install after a DooPlex loss needs it; it fits under the 3 GB cap) | leave it out: after a loss, no new install until a golden is rebuilt and vouched | the same | — | | D37 | R-1236 | Raise Gitea's memory limit from 1 GiB to 2 GiB? It was killed for memory 4 times in 17 days, once during today's release. | **Yes** (DooPlex has the RAM; a DooPlex change, so yours) | keep 1 GiB: a restart in release hours fails box updates and CI for a minute | the same | — | | D9 | R-1062 | Split the N100's USB drive in two, for multi-drive tests? | **No — keep one drive** (the lived-in box keeps its nightly whole-box backup) | Split: the box loses its local whole-box backup, or you buy a second drive first | it stays one drive; multi-drive tests stay on Tester 1 | — | @@ -49,7 +48,7 @@ Newest first. CC's pick is in the „Pick" column. Each row also has its full te 6. BIOS: on the N100, read „After Power Loss"; on the HP, set „Bypass F1 Prompt on Configuration Changes" to Enable. Since 2026-10-14. 7. Pin the first Facebook post (R-917), and look at the new cover in the Facebook phone app (R-919). Since 2026-10-12. -**2026-10-26.** On the hub's Hosts page, delete `Tester4-b5ec39` (CC's test VM, destroyed). If not: the hub alarms that the box went silent. +**2026-10-26.** On the hub's Hosts page, delete `Tester4-b5ec39` (CC's test VM, destroyed). If not: the hub alarms that the box went silent. After demo-hp's window has run (CC says when, Tuesday morning), revoke the token `cc-d55-window` (R-1294). **2026-10-23 do list.** 1. Print the break-glass sheet (`documentation/runbooks/break-glass-sheet.md`, the part from „The sheet”, one A4), fill it by hand, keep it outside the home (D32; R-923, R-924). If not: after a DooPlex loss no off-site copy opens. 2. On the hub's Hosts page: Discard appliance 49. If not: a dead entry alarms. diff --git a/documentation/architecture/09-update-architecture.md b/documentation/architecture/09-update-architecture.md index 2cbed3df..c9ab23c3 100644 --- a/documentation/architecture/09-update-architecture.md +++ b/documentation/architecture/09-update-architecture.md @@ -1099,6 +1099,15 @@ its length, and both fixes cost something the household would notice — operato `plex: security fix CVE-2026-96652,CVE-2026-96654,CVE-2026-96655,CVE-2026-96656 taken` and ended `done` in 30.1 s after the update's second-drive copy and undo copy. Released as controller 0.341.0, global floor 11:24 UTC. +### 2026-10-26 (night brief) — D55: demo-hp's one clean-up window gets a cap of 260 + +268. **demo-hp's one-shot clean-up window is granted with a raised cap of 260, not 220 (D55 = 260).** *Operator, + 2026-10-26 night.* Options: A) 260 — cost: one window may remove up to 260 old copies on a test box (the box still + removes only what its normal keep rule selects, and the fake-copy guard still runs); B) 220 as first ruled — cost: + the plan grows ~14 a night (209 on 2026-10-26, ~223 at the next run), so it is refused again. Why A: the cap must sit + above the honest plan on the night it is used. CC's token lacks `offsite:write`, so the operator made a one-purpose + token `cc-d55-window` (Off-site write only); CC used it once, 2026-10-26 19:10:31Z (`max:260` stored, read back). + ### 2026-10-26 (evening brief) — D53: held apps reach the operator; D54: demo-hp's stuck clean-up; R-1297 (B): a second CI runner 265. **A held app reaches the OPERATOR first, never the household (D53 = operator-only reminders first).** *Operator, diff --git a/documentation/backlog/OPEN-ITEMS.md b/documentation/backlog/OPEN-ITEMS.md index 2d052bfa..a5b09f93 100644 --- a/documentation/backlog/OPEN-ITEMS.md +++ b/documentation/backlog/OPEN-ITEMS.md @@ -211,7 +211,7 @@ stopping line that lies. | **R-1232** | Backup & restore | P2 | **A reinstalled box may take its first whole-box copy to ep0 up to a week late: the old box's copy in the same household space seems to count as its last one.** Test night 2026-10-23 (`audits/night-2026-10-23/P4-testnight/`): the new Tester4 box (guest 9201) had its PBS tier applied at 19:28 UTC, but no whole-box run to ep0 came all night; the only archive in the household's ep0 space is the rehearsal box's `ct/9201` of 2026-10-20 (same guest number), and the tier's cadence is 168 h. The agent's due-check reads the newest archive for that vmid on the tier's storage (`localapi/server.go` `newestArchiveOn`) — so the old box's copy reads as „2.6 days old, not due”. If so, the ep0 copy is the OLD box's for up to a week. PLAUSIBLE — read from the code and the night's silence, not proven by a log line (the controller logs the due-check only at debug). | OPEN | — | Prove: read `GET /backup/tiers` for the box (the agent's answer per tier) on the next reinstall; fix: a tier's „last backup” counts only archives this host wrote (or after the host's enrollment); an agent test with a foreign archive in the namespace | CC | | **R-1235** | Backup & restore | P3 | **The release copy on ep0 (D33) is proven to load, not yet to restore a registry — and it follows the newest versions, not the ones the fleet needs.** Built 2026-10-23 (`scripts/release-copy/`, `audits/day-2026-10-23/08-release-copy-D33.txt`). (a) `total-loss-of-dooplex.md` „Restore the release copy” steps 2 (skopeo push back, same digest) and 3 (agent packages back) were never walked into a real new registry — only a `docker load` on 9202. (b) The copy keeps the newest 3 image tags and 5 agent versions; it does not ask the hub for the floor and the vouched agent and always include them — a vouch more than 5 releases behind would fall out. (c) The golden (`felhom-golden`, 637 MB for 0.331.0) is not copied: a new install after a DooPlex loss needs it (the brief named images + agent packages only; one golden fits under the 3 GB cap). | OPEN | — | (a) a throwaway registry on demo-hp, push one tag back, compare digests; (b) read `/configuration`'s floor + vouched agent; (c) the operator says whether the golden goes in | CC | | **R-1263** | Backup & restore | P4 | **Small items from review round 2:** a LOCAL restore's put-back mails the operator „its off-site restore FAILED” (`felhom-controller/controller/cmd/controller/main.go:1514-1518`, shared via `restore_unit.go:517-518`); Remove — even „keep my data” — deletes undeclared volumes labelled with the app's project, which no tier holds (`internal/stacks/delete.go:763-764`, R-1117); the capped copy's `--delete` reads the source in the daemon's mount view, the emptiness check in the controller's (`tier2.go:834` vs `:464`); stale comments — `felhom-agent/internal/localapi/backup_cap.go:15` („keep-last”) and `restore_crash_putback.go:32` (names a test file that does not exist); the qBittorrent password goes to whatever answers the DNS name `qbittorrent` (`upwatch/explain.go:94,108-113`); kept-copy Remove has a check-to-remove gap (`tier2_kept.go:92-111`); the release-copy restore check trusts hashes stored on ep0 (`runbooks/total-loss-of-dooplex.md`); rrsync without `-munge`/`-wo` (`audits/day-2026-10-23/08-release-copy-D33.txt:6`). | OPEN | — | fix in each repo's next attended pass | CC | -| **R-1294** | Backup & restore | P3 | **demo-hp's weekly off-site clean-up was refused by its own guard on 2026-10-26 (window 16): the plan would remove 209 of 359 snapshots, above the week's cap (179) — nothing was deleted, and each refused week adds to the next plan.** Seen in the operator mailbox during the 2026-10-26 release run (03:17Z, `offsite_prune_guard_refused`, error; details `count_before 359, count_after 359, outcome guard-refused`). The cap refusal is kept on purpose (R-824, R-867 closed the false ones); the open question is whether a store that once exceeds the cap can ever prune again by itself, or refuses every week while it grows. demo-hp is the Tier-0 box with many hand-made backups (sweeps, probes). Earlier cap refusals: demo-hp 2026-10-03 (window 1), demo-felhom 2026-10-05 (window 3). | **WAITING-ON-OPERATOR** — 2026-10-26 evening: CC could NOT set the D54 grant — `POST /offsite/window-grant/demo-hp` (`max_remove=220`) with CC's hub token answered **403 „missing scope offsite:write"** (the hub log shows the refusal; nothing granted). The plan also grows: restic's dry-run 2026-10-26 18:56Z still 359 copies / plan 209 / default cap 179 (before tonight's run); a simulation of the keep rule adds ~14 per night (14 of 18 groups back up nightly), so ~223 at the night of 10-27 and ~237 at 10-28 — a cap of 220 would be refused again. STATUS D55: the operator grants it, with the cap. The grant opens the box's NEXT window request, which comes after its next nightly off-site run. Ruled D54 = prune once (`09` §3 decision 266). **Earlier:** WAITING-ON-OPERATOR — 2026-10-26 (read only, `audits/r1294-offsite-cap-2026-10-26/`): the plan is HONEST (restic's own dry-run: keep 150, remove 209; every removal 7–30 days old; every app keeps ≥ 9 or its last copies; the bulk is one test day, 2026-10-16, 152 snapshots). It can NOT recover by itself: the cap is half the store and grows ~49/week while the plan grows ~84/week. Only demo-hp has this shape (demo-felhom and Tester 1 pruned normally 10-26). The pick is STATUS D54. **Earlier:** OPEN | — | read the guard's cap rule (controller off-site prune) against demo-hp's snapshot list: is the plan honest (old, superseded) or not; if honest, say how such a store gets back under the cap (an attended one-time prune, or a cap that follows the store) — a design question for the operator if it changes what a box may delete | CC | +| **R-1294** | Backup & restore | P3 | **demo-hp's weekly off-site clean-up was refused by its own guard on 2026-10-26 (window 16): the plan would remove 209 of 359 snapshots, above the week's cap (179) — nothing was deleted, and each refused week adds to the next plan.** Seen in the operator mailbox during the 2026-10-26 release run (03:17Z, `offsite_prune_guard_refused`, error; details `count_before 359, count_after 359, outcome guard-refused`). The cap refusal is kept on purpose (R-824, R-867 closed the false ones); the open question is whether a store that once exceeds the cap can ever prune again by itself, or refuses every week while it grows. demo-hp is the Tier-0 box with many hand-made backups (sweeps, probes). Earlier cap refusals: demo-hp 2026-10-03 (window 1), demo-felhom 2026-10-05 (window 3). | **OPEN — the grant is SET (D55 = 260, `09` §3 decision 268)**: 2026-10-26 19:10:31Z through the operator's one-purpose token `cc-d55-window`, read back `max:260`; it is used at demo-hp's next window request (after its nightly off-site run, ~03:15Z). Close when that window prunes; then the operator revokes the token. **Earlier:** WAITING-ON-OPERATOR — 2026-10-26 evening: CC could NOT set the D54 grant — `POST /offsite/window-grant/demo-hp` (`max_remove=220`) with CC's hub token answered **403 „missing scope offsite:write"** (the hub log shows the refusal; nothing granted). The plan also grows: restic's dry-run 2026-10-26 18:56Z still 359 copies / plan 209 / default cap 179 (before tonight's run); a simulation of the keep rule adds ~14 per night (14 of 18 groups back up nightly), so ~223 at the night of 10-27 and ~237 at 10-28 — a cap of 220 would be refused again. STATUS D55: the operator grants it, with the cap. The grant opens the box's NEXT window request, which comes after its next nightly off-site run. Ruled D54 = prune once (`09` §3 decision 266). **Earlier:** WAITING-ON-OPERATOR — 2026-10-26 (read only, `audits/r1294-offsite-cap-2026-10-26/`): the plan is HONEST (restic's own dry-run: keep 150, remove 209; every removal 7–30 days old; every app keeps ≥ 9 or its last copies; the bulk is one test day, 2026-10-16, 152 snapshots). It can NOT recover by itself: the cap is half the store and grows ~49/week while the plan grows ~84/week. Only demo-hp has this shape (demo-felhom and Tester 1 pruned normally 10-26). The pick is STATUS D54. **Earlier:** OPEN | — | read the guard's cap rule (controller off-site prune) against demo-hp's snapshot list: is the plan honest (old, superseded) or not; if honest, say how such a store gets back under the cap (an attended one-time prune, or a cap that follows the store) — a design question for the operator if it changes what a box may delete | CC | ## Storage & devices — 13 rows (P3 4, P4 9) @@ -399,3 +399,4 @@ stopping line that lies. | **R-1297** | Process & tooling | P3 | **CI waits dominate a working day: felhom.eu's CI run takes ~13 min and the ONE runner makes every repo's job wait behind it (up to 10–17 min queued).** Measured 2026-10-26 (operator: „these CI waiting times are getting out of hand"): since 10-25 felhom.eu runs median 793 s, the controller/agent/catalog 57–102 s; queue median 5 s, max 548–1023 s. In run 2366, ~10 of 13 minutes are four suites — `test_gate_decoys.py` 232 s, `test_due_checks_gate.py` 133 s, `test_repo_gates.py` 132 s (it runs the whole gate set a SECOND time), `dooplex-offsite/test_dooplex_offsite.py` 69 s — plus `hub-gofmt` ~110 s (a cold Go cache per run). **Ruled 2026-10-26 by the operator: BOTH** — (A) trim felhom.eu CI: the heavy suites only when their own files change, no duplicate full run, a Go cache for hub-gofmt; (B) a second CI runner on DooPlex. | **OPEN — (A) BUILT 2026-10-26 evening, closes after the first nightly run (2026-10-27 03:20 UTC) is read.** Both runners proven at once (jobs 2374 + 2375 on `felhom-gates-runner` / `-2`, 17:11:41Z). felhom.eu CI, one push each: **(a) docs only** 810 s (old runner) → 351 s (2 CPUs) → see the close-out; **(b) a gate script** 351 → 203 s (`ecd82aba`, job 2376); **(c) a hub change** 351 → 118 s (`7e57e824`, job 2378). Heavy suites by path + tag + nightly; `test_repo_gates.py` and due-checks group G test the wiring (130 s each → under 3 s). Decoys through CI's entry point with the filter ON: a docs-only register break → red; an inert gate → GREEN at first (a decoy hole, fixed in `e33ec821`) → red. No Go cache: CI has no Go (the 110 s was security-scan, now 27 s). `audits/r1297-ci-trim-2026-10-26/`. (B) DONE by the operator (`09` §3 decision 267). | — | (A) CC, a run of its own (a CI scope change, ruled); (B) a DooPlex change — a brief or the homelab session; and CC waits on CI only before a release or at the end of a run | CC | | **R-1298** | Monitoring & notifications | P3 | **Scratch guest 9202 carries leftovers made outside the product, and D53 would remind about them every week once released: four crash-loop holds since 2026-09-24 for apps that are not installed (`chaoscrash`, `chaosoom`, `chaosoomb`, `gokapi`), and a PrivateBin `app.yaml` (2026-10-25) with no container that the product reads as „installed".** Found 2026-10-26 evening in the D53 live proof: the new reporter sent the four holds at its first tick and the test hub mailed four reminders („held 32 days"). The product's removal clears an install's hold (R-704); these apps were removed by hand in a chaos drill. On a household box the same shape would mail the operator weekly about an app that does not exist. | OPEN | — | decide whether the reporter (or the hub) skips an `unhealthy_stop` / `update_failed` / `putback_pending` hold whose app is not installed (a restore hold stays reported — operator-cleared by design, R-379); then clear 9202's four holds with `-clear-restore-hold` and remove PrivateBin through the product | CC | +| **R-1299** | Process & tooling | P4 | **The operator's one-shot clean-up grant (`POST /offsite/window-grant/`, optional `max_remove`) has no button on any hub page** — the operator could not find it on 2026-10-26 (D55) and had to make a one-purpose access token so CC could call the route. The route, its scope (`offsite:write`) and its event exist (R-833); only the form is missing. | OPEN | — | a small form on the Off-site page (customer, optional cap ≤ 500), with a render test for both branches | CC | diff --git a/documentation/backlog/STATUS-HISTORY.md b/documentation/backlog/STATUS-HISTORY.md index fb31b91e..548a079f 100644 --- a/documentation/backlog/STATUS-HISTORY.md +++ b/documentation/backlog/STATUS-HISTORY.md @@ -5,6 +5,14 @@ Moved here byte for byte on 2026-10-14 (night session), newest first, so that `S **Updated 2026-10-13 (night): demo-hp, demo-felhom and Tester 1 run agent 0.155.0; demo-felhom runs controller 0.310.0, the other two 0.309.0 until the floor is raised. The open-items list is at 127. Reports: `REPORT-poweroff.md`, `REPORT-plugmeter.md`, `REPORT-usenet-2026-10-12.md`, `REPORT-newapps-2026-10-12.md`, `REPORT-disk-evening-2026-10-12.md`, `REPORT-answers-2026-10-12.md`, `REPORT-m3-2026-10-11.md`, `REPORT-redesign-2026-10-10.md`, `REPORT-testers-2026-10-10.md`, `REPORT-register-shrink-2026-10-10.md`, `REPORT-new-apps-2026-10-10.md`, `REPORT-release-2026-10-10.md`, `REPORT-break-the-circle-2026-10-09.md`, `REPORT-dooplex-survival-2026-10-09.md`, `REPORT-day4-2026-10-09.md`.** +## Moved from STATUS on 2026-10-26 (night) + +**Updated 2026-10-26 (evening).** CI is faster: two runners work at once, and a hub or document push now takes about 2 minutes instead of 6–13. Held apps now reach you (built, not released): a „Held apps" list on the host page and the Hosts page, and one mail after a waiting time. One pick for you: D55. + +| D55 | R-1294 | CC's hub token may not grant demo-hp's one clean-up (the hub says „missing scope offsite:write"), so you grant it: hub → Off-site → demo-hp → one window with a raised cap. Which cap? | **260** (the old-copy list grows about 14 a night: 209 on Sunday, about 237 by Wednesday night's window; the box still removes only what its normal rule removes, and the fake-copy guard still runs) | 220 as first ruled: it is refused again, nothing is deleted, one error mail | demo-hp stays stuck and mails an error every Sunday | — | + +**Ruled:** D55 = 260 (operator, 2026-10-26 night; `09` §3 decision 268). The grant was set 19:10:31Z. + ## Moved from STATUS on 2026-10-26 (evening run) **Updated 2026-10-26 (afternoon).** A fresh install's whole-box off-site copy no longer fails silently: the cause (PBS's 5-second permission cache) is fixed on ep0, and the hub retries and mails you if it still fails (hub 0.172.0). Every box runs controller 0.345.0: the torrent rule leaves the household's alternative speed alone. Two picks wait for you (D53, D54).