148 lines
10 KiB
Markdown
148 lines
10 KiB
Markdown
# REPORT — hub v0.89.0: the two halves of decision D-c, plus the R-165 merge spike (2026-08-02)
|
|
|
|
**Overwritten** per the standing rule. The prior contents (R-168, the CI runner, same day) have their
|
|
durable record in `scripts/CHANGELOG.md` and `CONTEXT.md` S-8/S-9/S-10.
|
|
|
|
**Companion report:** `felhom-controller/REPORT.md` holds the controller side (v0.191.0/.1/.2), the
|
|
full red-proof table, the Hungarian copy, and the live evidence for all three flows. This file covers
|
|
the hub change, the documentation coupling, and **Part 3's spike**.
|
|
|
|
---
|
|
|
|
## 1. Baseline drift — recorded, because the task's §1 was wrong
|
|
|
|
The task targeted hub **v0.87.0 → v0.88.0**. On arrival `main` was at `8ef92a3f` with hub **v0.88.0
|
|
already shipped** (R-172, the WAL fix), not `d5774d318941`/v0.87.0. Target corrected to **v0.89.0**.
|
|
Highest register ID in use was **R-173**, not R-171.
|
|
|
|
## 2. Hub change (v0.89.0)
|
|
|
|
**One new event type, not two.** The task called for a new customer-facing type *and* a new
|
|
operator-only one. Reconnaissance found `disk_warning`/`disk_critical` already allowlisted here, with
|
|
Hungarian `customerMessages`, in the controller's `DefaultEnabledEvents` and behind a UI checkbox —
|
|
**and with no producer in any repo.** The operator chose to wire that inert pair rather than mint a
|
|
near-duplicate, so only the operator type is new.
|
|
|
|
| Change | File | Why |
|
|
|---|---|---|
|
|
| `+ "recovery_unit_capture_failed"` | `internal/api/handler.go` (`allowedEventTypes`) | without it the controller's POST 400s and the event vanishes |
|
|
| `+ "recovery_unit_capture_failed"` | `internal/notify/dispatcher.go` (`operatorOnlyEvents`) | **this** is what makes it operator-only; the allowlist does not, and v0.78.0 claimed otherwise and shipped the defect |
|
|
| `- customerMessages["disk_warning"]`, `- ["disk_critical"]` | `internal/notify/templates.go` | `FormatCustomerEmail` PREFERS the entry over the message, so a static template would discard the drive label and the free-space figures the controller now sends. Same reason `offbox_enlarge_blocked` and `disk_health_degraded` have none |
|
|
| `+ func IsOperatorOnly` | `internal/notify/dispatcher.go` | lets the `api` package pin BOTH registers in ONE test; checked separately, allowlisted-but-not-operator-only is invisible. Read-only — the register stays unexported so nothing can widen it at runtime |
|
|
| `REUSE.md` §5 "new event type" rewritten | `REUSE.md` | it told readers to always add a `customerMessages` entry, which is **wrong** for operator-only types and **harmful** for dynamic-message ones |
|
|
|
|
**Tests 574 → 579**, full suite green (`go build ./... && go vet ./... && go test ./...`), all five
|
|
`repo_gates.py` gates OK.
|
|
|
|
**Red-proof (Scenario G), demonstrated not argued:** removing `recovery_unit_capture_failed` from
|
|
`operatorOnlyEvents` fails two tests, one reading *"a customer was emailed the OPERATOR-ONLY
|
|
recovery_unit_capture_failed (customer@example.com)"*. The dispatch test runs under the **breaking**
|
|
configuration — the customer has the event enabled and an email set — because that is the only
|
|
configuration in which the missing entry is visible.
|
|
|
|
**Live (guest 9201 → hub):** both event types accepted and stored; `operator | sent`; and the positive
|
|
observable `customer | recovery_unit_capture_failed | skipped | operator_only` read from
|
|
`notification_log`. The customer half: `customer | disk_warning | sent` and `customer | disk_critical
|
|
| sent` with the dynamic Hungarian intact.
|
|
|
|
**Deploy:** GitOps only — `manifests/hub.yaml` bumped 0.88.0 → 0.89.0 (`6d359a5`), pushed, then a
|
|
deliberate ArgoCD hard-refresh + sync. Never `kubectl set image`. App `felhom` **Synced / Healthy**,
|
|
`deploy/hub` rolled out, running `gitea.dooplex.hu/admin/felhom-hub:0.89.0`, startup log clean.
|
|
|
|
## 3. Part 3 — the R-165 spike. **M1-M5 each answered; nothing was changed.**
|
|
|
|
Full document: `documentation/audits/SPIKE-r165-mp1-merge-2026-08-02.md`. No partition was created,
|
|
resized, moved or deleted; no golden rebuilt; no guest config edited. `ep0` and Peti's box were not
|
|
contacted (D-d, `runbooks/target-selection.md`).
|
|
|
|
**M1 — what is actually there. ANSWERED, and it contradicts the architecture doc.**
|
|
|
|
| | demo-felhom | demo-hp | golden default |
|
|
|---|---|---|---|
|
|
| `mp0` `/var/lib/docker` | **200 G** (13 G used) | **50 G** (5.4 G used) | 16 G |
|
|
| `mp1` `/mnt/sys_drive` | **50 G** (2.0 G used, 5%) | **20 G** (92 M used, 1%) | 8 G |
|
|
|
|
§7.5 documents the appliance as `mp0 50G / mp1 20G` — that is demo-hp exactly and **not** demo-felhom.
|
|
Any merge plan expressed as a fixed pair is already wrong for one of the two boxes that exist. §7.5's
|
|
headline bound (*"≈ 19 GB … ≈ 10 GB"*) is derived from `mp1 = 20 G` and is therefore one box's, not
|
|
the fleet's → **R-175**, filed and §7.5 annotated in this session.
|
|
|
|
**M2 — what lives on `mp1`. ANSWERED, and it is not only backups.** Four things would move:
|
|
Tier-1 units of driveless apps (269 M, ~30 apps on demo-felhom), **Tier-2 mirrors (1.7 G — i.e. the
|
|
MAJORITY is Tier 2, not Tier 1)**, the `userdata/import` drop zone which lives on the system drive by
|
|
**contract** (R-75), and the system-data userdata namespace. Observed fill is 5% / 1%: the constraint
|
|
is a **ceiling** problem, not a current-fill one.
|
|
|
|
**M3 — which merge shapes exist. ANSWERED for three shapes, with ONE item explicitly unmeasured.**
|
|
The golden **fails closed on the split in four places**, not one (`build-golden.sh:126,130`
|
|
separate-mount asserts + `:315,319` vzdump-exclusion guards). The archive scope `rootfs+mp0+mp1` stays
|
|
complete after a merge (the data moves onto `mp0`). `mountParity` holds for new archives. **Unmeasured
|
|
and reported as such:** whether a *pre-merge* archive restore-tests into a *merged* guest — reading
|
|
`mountParity` says it should, but that is reasoning from source about an unvalidated mechanism, which
|
|
this project has got wrong four times → **R-176**.
|
|
|
|
**M4 — the bulkhead. ANSWERED, and it is the important one.** `mp1` is not only a ceiling: today an
|
|
overflow is refused per app with the last good unit byte-identical **and cannot reach
|
|
`/var/lib/docker`**. After the merge it can, and a full Docker data-root is a stopped box, not a slow
|
|
one. Four replacements costed — a reserved block percentage, **a refusal threshold in the capture
|
|
path**, a project quota, or deeming R-167's warnings sufficient — with the trade-off of each.
|
|
**Deliberately not chosen: this is the operator's ruling.**
|
|
|
|
**M5 — existing boxes. ANSWERED for the measurable population; one part honestly UNMEASURED.** The
|
|
hub's `/hosts` register holds four hosts, **two ONLINE**, both demo boxes — and both are **Tier 0,
|
|
therefore reinstallable rather than migratable (D-d)**, so migration cost for the measurable population
|
|
is **zero**. **D-a's condition (1) — "before any external install" — is currently SATISFIED**, which
|
|
makes this the cheapest this decision will ever be. **`peti-felhom` exists as a customer with NO host
|
|
in the register**, so its layout is not knowable from the hub and the box was not contacted; whether it
|
|
needs converting or reinstalling is the operator's information. The in-place migration procedure has
|
|
**never been rehearsed**, so "is the box restorable at every point of it?" is currently unknown → also
|
|
**R-176**.
|
|
|
|
**Ranked options and recommendation:** (1) **S1 — one volume with the two paths as directories — plus
|
|
B2, a refusal threshold in the capture path**, shipped as a fresh-install shape with the demo boxes
|
|
reinstalled; (2) S1 + warnings only; (3) S3, grow `mp1` and keep the split (D-a's rejected baseline,
|
|
measured for comparison); (4) S2, two mounts on one pool — **not recommended at all**, it satisfies
|
|
every assertion while delivering none of the benefit and converts a clean per-app refusal into a
|
|
shared-pool exhaustion neither `df` can see coming.
|
|
|
|
**STOPPED at the operator's question**, per the task. The merge is next session's supervised work.
|
|
|
|
## 4. Documentation coupling
|
|
|
|
| File | Change |
|
|
|---|---|
|
|
| `documentation/backlog/OPEN-ITEMS.md` | **R-158** closed (by R-167 — *no second row for the same wire*); **R-167** closed; **R-165** updated with M1-M5 + the operator question, stays open; **4 new rows** R-174/175/176/177 |
|
|
| `documentation/backlog/ROADMAP.md` | R-158 collapsed to a shipped one-liner; R-167 added as shipped; R-165 added as spiked/waiting-on-operator |
|
|
| `documentation/architecture/00-capability-map.md` | **two new rows**, both **PROVEN-LIVE** with live citations |
|
|
| `documentation/architecture/07-backup-architecture.md` §7.5 | **S-1: the contract changed in the same session.** The section's closing claim *"nothing warns when an app crosses the line"* is now false; the alerting is written in, and the one-box-vs-fleet caveat added |
|
|
| `CONTEXT.md` | **S-11** (D-c's routing, and why R-158's own `backup_failed` proposal was overruled) and **S-12** (the monitoring landed *before* the merge, not with it) |
|
|
| `STATUS.md` | new plain-language section; the merge decision added to *Waiting on you*; **two older entries trimmed** so the page did not grow — one screen, per its own rule |
|
|
| `REUSE.md` | the "new event type" extension point rewritten (see §2) |
|
|
|
|
## 5. Register IDs
|
|
|
|
**Opened:** R-174, R-175, R-176, R-177. Each established free by
|
|
`grep -ro "R-17n\b" documentation/ *.md` → **0 hits**, run before minting.
|
|
**Closed:** R-158, R-167, R-174. **Updated, still open:** R-165, R-163 (unchanged — it stays the
|
|
record of the constraint until the merge lands).
|
|
|
|
## 6. CI — run ids and conclusions
|
|
|
|
Checked by PULL from `…/actions/tasks`, matching `head_sha` to each commit — CI emails only on
|
|
failure, so a green that was never looked at is an assumption, not an observation. **Every commit this
|
|
session, both repos, is green.**
|
|
|
|
| Repo | Commit | Task id | Run # | Conclusion |
|
|
|---|---|---|---|---|
|
|
| `felhom-controller` | `cf48214` (v0.191.0) | 31 | 11 | **success** |
|
|
| `felhom-controller` | `5adae4d` (v0.191.1) | 34 | 12 | **success** |
|
|
| `felhom-controller` | `9a3c485` (v0.191.2) | 35 | 13 | **success** |
|
|
| `felhom.eu` | `179dd79` (hub v0.89.0) | 32 | 17 | **success** |
|
|
| `felhom.eu` | `6d359a5` (manifest 0.89.0) | 33 | 18 | **success** |
|
|
| `felhom.eu` | `41dbecb` (docs) | 36 | 19 | **success** |
|
|
|
|
## 7. `--no-verify`
|
|
|
|
**Not used anywhere.** Every push in this session ran `.githooks/pre-push` (`repo_gates.py --fast` /
|
|
`controller_gates.py --fast`) and passed.
|