admin/felhom.eu
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

E-2d executed on a fresh box: C1/C2 proven, C3/C4 partial, C5 FAILS — R-112/113/114

Browse Source 
Full ISO/PAIRING route on a nested PVE VM on demo-hp, after R-111 was fixed
earlier in the session. Bind -> running controller in 3m35s. The install fetched
the artifacts published an hour before and restored the golden baked 20 minutes
before, so the publish train is proven end to end on a real install.

C1 PROVEN: "felhom-host-install v1.22.0", "Day-0 provision SUCCESS", guest 9201
running, bootstrap unit wrote its done-flag and self-disabled. This retires E-2's
"installer-logic-tested, not install-tested".

C2 PROVEN: both DEGRADED warning lines verbatim, backup.local_backup_target=local,
no felhom-backup storage created, and the install did not abort.

C3/C4 PARTIAL and C5 FAILED — three findings, none fixed:

R-112 (P1): E-2's degraded banner and offer have NO UI CONSUMER. The endpoint
returns byte-exact copy; grep 'backup-target' across every html/js/css is 0 hits
and no page handler injects the state. Templates fetch 18 distinct /api/storage/*
endpoints; these two are the only ones with zero references. v0.185.1 fixed the
router mount and stopped one layer short of the render. Fifth instance of
seam-built-but-never-wired.

R-113 (P1): the drive-absent gate CANNOT FIRE on device loss. planDriveGates
reads presence from BoundUnderParent = "is this path in the guest's mountinfo".
The raw mount is a device-bound systemd unit and dies with the device; the
agent's own bind is not device-bound and outlives it, so the gate sees "present"
forever. Live: agent reported the drive absent every 20s for 4.5 minutes, the
controller logged 0 [gate] lines, the hub received zero events -- neither
backup_target_absent nor the generic storage_disconnected. Sixth instance of the
class: E-2b wired the seam to a condition that cannot occur.

R-114: on target-drive loss the message claims the backup is on the system disk
(false) and offers the drive that just vanished. Invisible only because of R-112,
so it must be fixed BEFORE R-112 is wired.

Also filed as a second instance under R-110 rather than a new ID: host-install
fetches nine files from raw/branch/main and the hub vouches a sha for one;
E-2a's wrapper is installed 0755 to /usr/local/sbin, root-fenced in sudoers,
validated only by bash -n.

C4 is fully proven at API level: decline path (registration confers no role),
restart_required:true, agent did NOT self-restart (in-flight check performed and
recorded first), E-2a wrapper created the storage at the drive's own mountpoint,
and healthy renders nothing.

Teardown: VM destroyed, scratch storage removed, pvesm status after == before
(local-lvm 38.77%), guest 9201 and drill-r50 untouched. Hub records for e2d-fresh
remain -- delete correctly refused at four gates, finally "host is ONLINE";
deletable once it ages to DOWN. Command recorded in OPEN-ITEMS.md.

capability-map NOT touched: the customer-facing legs are broken rather than
proven, and the map has no E-2 rows at all.
This commit is contained in:
authentik Default Admin
2026-07-29 13:13:56 +02:00
parent 3dff3573f7
commit f3975cf5bc
4 changed files with 307 additions and 211 deletions
Download Patch File Download Diff File Expand all files Collapse all files
REPORT-e2d.md
+71 -42
View File
@@ -1,58 +1,87 @@
# REPORT — E-2d fresh-VM run: STOPPED at Phase 0 (2026-07-29)
# REPORT — R-111 fixed, then E-2 proven on a fresh box (2026-07-29)
`RUNBOOK-e2d-fresh-vm-2026-07-29.md`, executed by CC on DooPlex. Full evidence:
`documentation/audits/E2D-fresh-vm-2026-07-29.md`. Written as `REPORT-e2d.md` per the runbook §8.5;
root `REPORT.md` untouched.
Two phases in one session. Full evidence: `documentation/audits/E2D-fresh-vm-2026-07-29.md`.
Root `REPORT.md` untouched.
**Outcome: no VM created, no install run, no box touched, no teardown needed.** The run stopped at the
Phase 0 gate, per §3 (*"If any gate fails, STOP and report — do not adapt around it"*).
## Phase 1 — R-111: the Day-0 channel now serves the current software
## Why
A Phase 0 gate earlier the same day stopped the E-2d run before any VM existed: a fresh box would
have installed **agent 0.96.0 + controller 0.161.0**, ~17 and ~24 releases behind `main`.
`felhom-host-install.sh` does not install what is on `main`. It resolves the hub-vouched manifest
(`:423-436`) and fetches **Gitea generic packages** (agent `:1945`, golden `:2573`). Gitea's newest are
**agent `0.96.0`** and **golden `0.161.0`**; the hub's saved manifest selects exactly those; the global
controller floor is `v0.156.0`, below the golden's 0.161.0, so no self-update follows.
| Component | Fresh install gets | `main` / demo boxes |
| | Before | Now |
|---|---|---|
| host-install | **1.22.0** | 1.22.0 |
| agent | **0.96.0** | 0.113.0 — **never published**, direct-deployed |
| controller | **0.161.0** | 0.185.1 |
| agent (Gitea generic) | 0.96.0 | **0.113.0**, sha `5f3247f7…`, round-trip verified |
| golden (Gitea generic) | 0.161.0 | **0.185.1**, sha `dba00f3e…`, embeds controller 0.185.1 |
| hub `min_agent` | 0.93.0 | **0.113.0** (what controller v0.185.0 declares) |
## Claims
Bake clean on every marker: `Result=success`, overlay2, **all three mounts in the archive**, 0
FATAL/exclusions, HTTP 201, token-leak grep 0. GL-1 teardown: guest 9100 purged, secrets shredded,
drill disk restored to `virgin`. Agent + golden moved in **one** manifest POST so it never vouched a
new agent against an old golden. `min_agent` verified zero-impact first (all three enrolled hosts
already at 0.113.0). Global floor deliberately **not** raised — the golden now bakes 0.185.1.
| Claim | Verdict | Pinned to |
|---|---|---|
| C1 — real rc=0 install of 1.22.0 | **ACHIEVABLE, not run** | installer-side |
| C2 — Case B fires naturally | **ACHIEVABLE, not run** | `felhom-host-install.sh:627`, `:653-655` |
| C3 — degraded banner renders | **BLOCKED** | controller v0.185.1 `cdaeb36`; copy v0.185.0 `3f7cf2a` |
| C4 — offer moves the target | **BLOCKED** | controller v0.185.1 + agent v0.113.0 `58b598b` |
| C5 — `backup_target_absent` e2e | **BLOCKED** | controller v0.184.0 `c1a63de` + agent v0.112.0 |
Commit `3dff357`.
C3 unblocks by raising the hub floor to ≥0.185.0 (controller 0.185.1 **is** in the registry); measured
fleet impact nil — both demo boxes already run 0.185.1, Peti is DOWN 14 d and already below the current
floor. C4/C5 need agent 0.112.0/0.113.0 **published**, which runbook §0 forbids this run from doing.
## Phase 2 — the E-2d run, full ISO/PAIRING route
## Filed
Nested PVE VM on demo-hp, one disk, outside the `felhom` pool. Bind → running controller in
**3 m 35 s**. The install fetched exactly the artifacts published an hour earlier and restored
`vzdump-lxc-9100-2026_07_29-12_37_56` — the golden baked 20 minutes before. The publish train is
proven end to end on a real install.
**R-111 (P1)** — the Day-0 artifact channel is 17 agent releases stale. v0.97.0–v0.113.0 unpublished,
stranding the entire R-82 tiered-backup arc plus **F-CRIT-2** and **F-REBOOT**. A new customer's box
installs without them. Mirror of R-110, not a duplicate: R-110 = publishes instantly with no staging;
R-111 = the publish gate exists and was never walked.
| Claim | Verdict |
|---|---|
| **C1** host-install 1.22.0 completes a real install, rc=0 | ✅ **PROVEN** |
| **C2** Case B fires naturally | ✅ **PROVEN** — both DEGRADED lines verbatim, `local_backup_target=local`, install did not abort |
| **C3** degraded banner renders **to a customer** | ⚠️ **PARTIAL** — API byte-exact; **no UI consumer exists** → **R-112** |
| **C4** offer appears and moves the target | ⚠️ **PARTIAL** — decline path, `restart_required:true`, no self-restart, E-2a wrapper, healthy-renders-nothing all PROVEN at API level; offer equally invisible → **R-112** |
| **C5** `backup_target_absent` end to end | ❌ **FAILED** — zero events on any channel → **R-113** |
## The three findings
**R-112 (P1)** — E-2's banner and offer have **no UI consumer**. The endpoint returns byte-exact copy;
`grep 'backup-target'` across every `*.html`/`*.js`/`*.css` → **0 hits**, and no page handler injects
the state. Decisive contrast: templates fetch **18** distinct `/api/storage/*` endpoints;
`backup-target` and `backup-target/assign` are the only two with zero references. v0.185.1 fixed the
router mount and stopped one layer short of the render. Fifth instance of seam-built-but-never-wired.
**R-113 (P1)** — the drive-absent gate **cannot fire on device loss**. `planDriveGates` reads presence
from `BoundUnderParent` = "is this path in the guest's mountinfo". The raw mount is a device-bound
systemd unit and dies; **the agent's own bind is not device-bound and outlives the device**, so the
gate sees "present" forever. Live: agent said `enrolled drive absent by UUID` every 20 s for 4½
minutes, controller logged **0** `[gate]` lines, hub got **zero** events — neither the specific nor the
generic one. Sixth instance of the class, one layer deeper: E-2b wired the seam to a condition that
cannot occur.
**R-114** — on target-drive loss the message says the backup is *"on the same disk as the system"*
(false) and offers **the drive that just vanished**. Invisible today only because of R-112 — so
**R-114 must be fixed before R-112 is wired.**
Also filed as a **second instance under R-110** (not a new ID): host-install fetches **nine** files
from `raw/branch/main` and the hub vouches a sha for **one**; E-2a's wrapper is installed 0755 to
`/usr/local/sbin`, root-fenced in sudoers, validated only by `bash -n`.
## Record
- `OPEN-ITEMS.md` — **R-111 opened** (READY (M), P1). **E-2d re-stated**, not closed: the attempt, the
blocker, the C1/C2-vs-C3/C4/C5 split, and every Phase 0 answer so a resumed run does not re-derive them.
- `ROADMAP.md` — R-111 filed under **P1 — closed-alpha blockers**.
- `architecture/00-capability-map.md` — **not touched.** Nothing was proven live; no row qualifies.
- `OPEN-ITEMS.md` — **R-112/R-113/R-114 opened**; E-2d re-stated with results and left open for the
residue; E-2's "NOT yet live-proven" list resolved into proven / known-broken; R-94 fully unblocked;
R-110 extended; a WATCHING row for the drill cleanup.
- `ROADMAP.md` — R-112/R-113/R-114 under P1; R-111 marked SHIPPED.
- **`architecture/00-capability-map.md` not touched** — for two reasons: the customer-facing legs are
broken rather than proven, and the map has **no E-2 / backup-target rows at all** (worth noting
against the ROADMAP's coupling rule).
## Not done
## Teardown
C1–C5 all unproven. No VM, no ISO boot, no appliance registered/bound/discarded, no storage created or
modified on demo-hp, no drive attached or detached, no hub setting written (the floor was read only),
no code written.
VM destroyed, scratch storage removed, **`pvesm status` after == before** (`local-lvm` 38.77 %,
byte-identical), guest 9201 and drill-r50 untouched. **Hub records for `e2d-fresh` remain** — the
delete was attempted and correctly refused at four gates, finally `host … is ONLINE`; it becomes
deletable once the destroyed host ages to DOWN (>1 h). Command recorded in `OPEN-ITEMS.md`. The stale
2026-07-25 appliance (`206c8838…`) was left alone — not mine.
The runbook's **§5.1a operator STOP is retired**: `HUB_PW` is in `~/.config/credentials` and hub auth
was verified working, so CC can perform the bind itself on a resumed run.
## One human step, and a premise correction
The runbook's §5.1a operator STOP (the bind) is **retired** — CC did it. But E-2d's premise that a
fresh install yields a CC-drivable claimable customer is **wrong**: the claim code is bcrypt-hashed and
email-only, and the gate covers everything except `/claim`, `/api/health`, `/static/`. One operator
relay of the emailed code was required — which also proved the claim flow end to end.