docs(runbooks): onboarding draft v3 -> v4 — post-ship refresh (R-36/R-39 workarounds deleted, HP t740 second datapoint)
Per the 2026-07-21 refresh brief: R-39 interim blocks (B4/E1) and the R-36 manual-Save block (C4) deleted — both shipped and proven live; freemail.hu gate proven (R-4 COMPLETE); golden/floor-lift note now cites two shapes (rehearsal + virgin HP t740 day-0 lift 0.153.0->0.156.0); A3 loader table per operations/nodes.md (N100=mkimage/SB-off per record, HP t740=shim/SB ENABLED); B2 multi-NIC cabled-port gotcha (R-59/R-60 pending); new A5 gate (agent >=0.93.0 deployed box-side before the first escrow ceremony); D offboarding pointer to §G (R-25b). DRAFT status and the C7 graduation gate unchanged. ROADMAP R-25b pointer follows the rename. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UuFPHmHNrCJj1VhY6QdDMU
This commit is contained in:
@@ -84,7 +84,7 @@
|
||||
| R-26 | **Guided old-history recovery via a retained superseded escrow + the recovery code.** Enabled by hub v0.60.0 (Part B) which now RETAINS superseded escrow blobs (`host_escrow_superseded`, `ListSupersededEscrow`). Build the flow that, given the customer's recovery code, unwraps a retained old blob → recovers the old repo passphrase → mounts/reads the moved-aside `.orphaned-<date>` repo for restore. | M | idea (enabled by v0.60.0) | Turns "history recoverable in principle" into a real customer-drivable path; pairs with the controller v0.142.0 orphaned-repo move-aside. Origin `DIAGNOSE-offbox-repo-orphaned-2026-07-17` |
|
||||
| R-27 | **Customer-facing self-bind page (R-21 slice C follow-on).** Today an unclaimed appliance is bound by the OPERATOR on the Hosts page (hub v0.62.0). Build the customer-facing flow so a customer can claim/bind their own freshly-installed box. | M | **SHIPPED (slice 1, hub v0.66.0, 2026-07-17)** | **Slice 1 = the FIRST-box flow, DONE:** operator "Send self-bind link" → 7-day tokenized capability link over Hungarian email → public two-factor `/bind/<token>` (console pairing code shown on the box screen + retrieval passphrase) → hub stages the bind via the same `BindAppliance`, no operator. Viktor's 3 rulings verbatim (console code not a list; operator-sent link; 5-attempt lockout→call support). No oracle; expiry falls back to operator-bind; THE TRAP `/bind/` exemption tight (single predicate, 4 red-proofs). GC verdict §3 (no appliance GC → the 7-day TTL stands alone). **Slice 1 PROVEN-LIVE 2026-07-18:** in the rehearsal the operator minted + emailed the link at 16:28:55 and **the customer bound their own box at 16:29:55 with zero failed attempts**, `appliance_bound` carrying source `customer_selfbind`, credential delivered 26 s later with no operator action (`tests/VALIDATION-n100-rehearsal-2026-07-18.md`). The new-ISO console banner rendered on the real box. **Slice 2 = console-passphrase bind → R-27c** (below). Reused the appliance_registrations + one-shot delivery machinery; the capability link is the whole customer-auth surface (no login system built). Controller/agent untouched. See hub v0.66.0 REPORT. **Multi-box per link is out of scope by design** (single-active, one-shot → binding several boxes = repeated operator sends) |
|
||||
| R-27b | **Customer self-bind, second-box flow (controller side).** For a customer who ALREADY has a bound box and installs another, the controller shows a dismissable "bind another box" prompt (and a bind-later entry under settings) that walks to the hub `/bind/` page — so a returning customer isn't emailed a fresh operator-sent link for every box. Mechanism sketched in the hub v0.66.0 REPORT; NOT built (R-27 slice 1 deliberately did not touch the controller). | M | idea (minted by hub v0.66.0) | Origin: hub v0.66.0 slice-1 ship (first-box only). Reuses the same `/bind/` public page + tokenized-link machinery; adds a controller-side entry point + the operator "mint a link for an existing customer" affordance |
|
||||
| R-25b | **RULED: customer DELETE becomes a guided full-teardown cascade.** The middle-tier Customer RESET (hub v0.61.0) runs the full external teardown (Hetzner sub-account/box + PBS namespace/groups/token) and refuses while any host row exists. The Danger-zone DELETE still (a) leaves host rows and (b) does NOT run that teardown. | M (was S) | **SHIPPED hub v0.69.0 (2026-07-21)** | **operator ruling 2026-07-21**: **DELETE subsumes the whole cascade, behind explicit consent.** Three separate acknowledgements, each its own checkbox — (1) the host(s) will be deleted, (2) the customer will be RESET including external teardown and offsite data destruction, (3) the customer record and escrow will be purged — plus a **typed customer-name confirmation** before the button arms. Internal order is **host-delete → RESET → delete**, which preserves every existing invariant rather than relaxing any: RESET keeps its no-hosts precondition (hosts are already gone by then), and escrow keeps its demote-then-purge custody rule (host delete DEMOTES to retained custody, the final delete PURGES — the one true purge point). **Re-sized S → M: this is a multi-step destructive wizard with three acks and a typed confirmation, not a checkbox.** Implementation is explicitly NOT part of TASK-E; the row carries the ruling and awaits its own spec. **It no longer blocks R-3** — the model is decided, so the friend-alpha runbook can be written against it. **IMPLEMENTED per the ruling (TASK-I, hub v0.69.0):** `POST /configs/{id}/delete` now runs `hosts → RESET → purge`; three acks + typed customer-id + a stale-preview check + the ONLINE-host refusal, all gates before any write (zero side effects on refusal); custody purged exactly ONCE in leg 3 (leg 2 runs with `purgeEscrow=false`); ruling-3 preserved BY CONSTRUCTION and asserted from inside leg 2; failed legs retain the journal and the dialog offers Resume. Standalone RESET byte-identical. 5 red-proofs. Offboarding guidance: `runbooks/RUNBOOK-onboarding-draft-v3.md` §G. **v0.70.0 follow-up (same day, found validating against the live hub):** a completed delete still left the customer on the Customers list and still ALERTING, because `GetCustomers()` is report-derived and no tier ever deleted a report — new **residue** leg (reports/telemetry/log-tails/notif-prefs + the credential-bearing `appliance_registrations`/`selfbind_tokens`), and **ghost customers are now deletable** (404 = nothing here, not no-config-row) |
|
||||
| R-25b | **RULED: customer DELETE becomes a guided full-teardown cascade.** The middle-tier Customer RESET (hub v0.61.0) runs the full external teardown (Hetzner sub-account/box + PBS namespace/groups/token) and refuses while any host row exists. The Danger-zone DELETE still (a) leaves host rows and (b) does NOT run that teardown. | M (was S) | **SHIPPED hub v0.69.0 (2026-07-21)** | **operator ruling 2026-07-21**: **DELETE subsumes the whole cascade, behind explicit consent.** Three separate acknowledgements, each its own checkbox — (1) the host(s) will be deleted, (2) the customer will be RESET including external teardown and offsite data destruction, (3) the customer record and escrow will be purged — plus a **typed customer-name confirmation** before the button arms. Internal order is **host-delete → RESET → delete**, which preserves every existing invariant rather than relaxing any: RESET keeps its no-hosts precondition (hosts are already gone by then), and escrow keeps its demote-then-purge custody rule (host delete DEMOTES to retained custody, the final delete PURGES — the one true purge point). **Re-sized S → M: this is a multi-step destructive wizard with three acks and a typed confirmation, not a checkbox.** Implementation is explicitly NOT part of TASK-E; the row carries the ruling and awaits its own spec. **It no longer blocks R-3** — the model is decided, so the friend-alpha runbook can be written against it. **IMPLEMENTED per the ruling (TASK-I, hub v0.69.0):** `POST /configs/{id}/delete` now runs `hosts → RESET → purge`; three acks + typed customer-id + a stale-preview check + the ONLINE-host refusal, all gates before any write (zero side effects on refusal); custody purged exactly ONCE in leg 3 (leg 2 runs with `purgeEscrow=false`); ruling-3 preserved BY CONSTRUCTION and asserted from inside leg 2; failed legs retain the journal and the dialog offers Resume. Standalone RESET byte-identical. 5 red-proofs. Offboarding guidance: `runbooks/RUNBOOK-onboarding-draft-v4.md` §G. **v0.70.0 follow-up (same day, found validating against the live hub):** a completed delete still left the customer on the Customers list and still ALERTING, because `GetCustomers()` is report-derived and no tier ever deleted a report — new **residue** leg (reports/telemetry/log-tails/notif-prefs + the credential-bearing `appliance_registrations`/`selfbind_tokens`), and **ghost customers are now deletable** (404 = nothing here, not no-config-row) |
|
||||
| R-25 | **Device-node TOCTOU hardening (drive init).** Graduate the controller v0.141.0 Observation: the `format → resolveEnrollUUID(path) → AssignDisk(uuid)` sequence has a narrow /dev-re-enumeration window (agent-guarded on the destructive format via anti-retarget durable-id; benign fs-UUID mount). Bind resolve+assign to the format's durable-id so the mount can't target a moved node. | S | idea | From the v0.141.0 F6 commit's security-review finding (`felhom-controller` REPORT). Low real risk (single-operator, agent-guarded), but cheap to close |
|
||||
| R-24 | **Guest RAM resize (live) — SHIPPED (agent v0.90.0 + controller v0.143.0, 2026-07-17).** The customer right-sizes the guest's memory from the controller's Rendszer page; the agent enforces every bound fresh + applies via PVE `SetConfig` (live cgroup, no reboot, Phase-0 proven). **Framing note:** the original hub-desired-state framing is SUPERSEDED by Viktor's controller-direct ruling (2026-07-17) — the resize is controller→agent local-API, never through the hub. Memory only (cores stay observation). **COMPLETE — the apply is proven in BOTH directions on metal (2026-07-18):** customer zero shrank the guest **11675 → 8192 MB (16:50:22)** and grew it **8192 → 12288 MB (17:02:17)**, each a live cgroup apply with no reboot, the new total rippling into the deploy page's memory math. The nested demo had proven the refusals; the N100 proved the applies. `tests/VALIDATION-n100-rehearsal-2026-07-18.md` ledger 9. **Cores/live-resize as hub desired-state is NOT built** (deferred, low demand). | M | **SHIPPED + PROVEN-LIVE** | See felhom-agent + felhom-controller REPORTs; capability-map row "customer right-sizes guest RAM". |
|
||||
| R-12 | Cluster mode: agent-follows-guest, bind-mount reconciliation on HA migration | XL | idea | Scoped 07-15; interim = HA-group pin to one node. Driven by Peti's two-node cluster |
|
||||
|
||||
+74
-46
@@ -1,41 +1,59 @@
|
||||
# DRAFT R-3 — Friend-alpha onboarding runbook (operator)
|
||||
|
||||
> **Status: DRAFT, post-rehearsal — all `[REFINE]` slots filled 2026-07-18.** Generalized from
|
||||
> `runbooks/RUNBOOK-rehearsal-v3.md` (S1–S8) and the shipped claim/escrow/self-bind flows, then
|
||||
> refined against the N100 rehearsal that executed it
|
||||
> (`tests/VALIDATION-n100-rehearsal-2026-07-18.md`). Every step here has been walked on real
|
||||
> hardware **except C7**, the customer-performed restore.
|
||||
> **Status: DRAFT, v4 — refreshed 2026-07-22.** Generalized from
|
||||
> `runbooks/RUNBOOK-rehearsal-v3.md` (S1–S8) and the shipped claim/escrow/self-bind flows, refined
|
||||
> against the N100 rehearsal that executed it (`tests/VALIDATION-n100-rehearsal-2026-07-18.md`),
|
||||
> and now carrying a **second full onboarding datapoint on virgin hardware** — the HP t740
|
||||
> (`demo-hp`, 2026-07-21; `operations/nodes.md`, `CONTEXT.md`). This refresh deleted the R-36 and
|
||||
> R-39 interim workarounds (both shipped and proven) and updated the freemail/loader/golden notes.
|
||||
> Every step here has been walked on real hardware **except C7**, the customer-performed restore.
|
||||
>
|
||||
> **It graduates to `RUNBOOK-onboarding.md` when C7 has evidence** — that step is simultaneously the
|
||||
> last unwalked step of this script and the capability map's last MISSING customer-facing row, so
|
||||
> one execution closes both. Until then this stays a DRAFT.
|
||||
>
|
||||
> Scope: **appliance hardware only** (BYO track is R-1/Peti, explicitly out of alpha scope).
|
||||
> Three steps below carry **interim workarounds for known defects** (R-31, R-36, R-39) — they are
|
||||
> marked inline and should be deleted from this runbook the moment those items ship.
|
||||
> One step below still carries an **interim workaround for a known defect** (R-31) — it is marked
|
||||
> inline and should be deleted from this runbook the moment that item ships.
|
||||
|
||||
## A. Before the visit (operator, ~30 min, all remote-doable)
|
||||
|
||||
1. **Hub:** create the customer (name, e-mail, domain `<név>.felhom.eu`) — verify the CF tunnel +
|
||||
zone token provisioning completes. Confirm the Day-0 manifest is current (agent, golden,
|
||||
MinAgent) and the managed-update floor is at the intended release — a stale golden is fine
|
||||
(the floor lifts the box post-install), a stale FLOOR is not. *The rehearsal proved the lift:
|
||||
a 0.143.0 golden reached 0.145.0 in **5 s**, unattended, inside the first minute of controller
|
||||
life.* Even so, **rebuild the golden before the first real tester** — proven-recoverable is not
|
||||
a reason to ship stale.
|
||||
(the floor lifts the box post-install), a stale FLOOR is not. *The current golden (0.153.0)
|
||||
carries all four infra images, and the floor-lift is now proven on TWO shapes:* the rehearsal
|
||||
(0.143.0 → 0.145.0 in **5 s**, unattended — `tests/VALIDATION-n100-rehearsal-2026-07-18.md`)
|
||||
AND a virgin board on first contact (0.153.0 → 0.156.0 during day-0, HP t740, 2026-07-21 —
|
||||
`CONTEXT.md`). Golden freshness before the first real tester is the standing rule in **F** —
|
||||
apply it there, don't re-derive it here.
|
||||
|
||||
**Mint the self-bind link now** ("Send self-bind link" on the customer's Setup tab). It is valid
|
||||
for **7 days**, and doing it here means the console banner's „e-mailben kapott link" is already
|
||||
true when the customer first looks at the screen. See B3.
|
||||
**Verify the self-bind link went out.** Since hub v0.67.0 (R-36) the link is **auto-minted at
|
||||
customer creation and at RESET completion**, so the console banner's „e-mailben kapott link" is
|
||||
already true by construction; "Send self-bind link" on the Setup tab is the manual re-mint if
|
||||
the customer lost it or the 7-day token expired. See B3.
|
||||
2. **Hardware:** the standard appliance + its external HDD. One-time bench prep list:
|
||||
memtest pass, SMART clean, BIOS queue for the single visit in B2.
|
||||
3. **Install media:** the **reusable generic pairing ISO** (secret-free, slice C). Only rebuild it
|
||||
when `scripts/` ships a new version — never per-customer. Flash via dd/Rufus-DD.
|
||||
Loader note per firmware: mkimage requires **Secure Boot OFF** (F1); record the board model +
|
||||
loader choice in the customer record.
|
||||
Loader per firmware — record the board model + loader choice in the customer record; full
|
||||
fleet inventory in `operations/nodes.md`:
|
||||
|
||||
| Board / firmware | Loader | Secure Boot |
|
||||
|---|---|---|
|
||||
| N100 (AMI AN3PLUS) | `mkimage` | **OFF** — that firmware can't relocate the signed GRUB (F1, `tests/VALIDATION-n100-baremetal-2026-07-16.md`) |
|
||||
| **HP t740 (AMI M42)** | **`shim`** | **ENABLED** (proven 2026-07-21, `operations/nodes.md`) |
|
||||
|
||||
**SB-off is a per-firmware workaround, not a Felhom requirement** — the shim loader boots with
|
||||
Secure Boot enforcing on firmware that handles it (the HP proved it).
|
||||
4. **Paper:** the tester one-pager (R-11, printed or PDF), the tester agreement
|
||||
(PETI-agreement sibling), and the recovery-code envelope step rehearsed in your head — the
|
||||
escrow ceremony is the one moment that must not be rushed.
|
||||
5. **Agent train ≥ v0.93.0 on the customer's box before the FIRST real tester's escrow ceremony.**
|
||||
v0.93.0 removes the four hyphenated EFF words from the recovery-code generator
|
||||
(`felhom-agent/CHANGELOG.md` v0.93.0, 2026-07-21) — ceremonies run **box-side**, so the fix
|
||||
protects a ceremony only once the train is **BUILT + DEPLOYED to that box**; source-only does
|
||||
not count. Already-issued codes stay valid.
|
||||
|
||||
## B. Install + bind (~15 min wall-clock)
|
||||
|
||||
@@ -45,18 +63,29 @@
|
||||
**pairing-code banner**; the box appears under **Unclaimed appliances** at the hub
|
||||
(verify uuid/MAC/SSH fingerprints match the physical box before binding — never bind a
|
||||
fingerprint you didn't read off the console).
|
||||
|
||||
> **Multi-NIC gotcha: verify the CABLED port got the DHCP lease during the install.** On the
|
||||
> HP t740 the cable was on the 4-port expansion card, which gets **no lease** — and instead of
|
||||
> aborting, the installer baked its `192.168.100.2` fallback as a **static** config and
|
||||
> completed: a box that looked installed and could never call home
|
||||
> (`operations/nodes.md` — the NIC map and the trap; filed as **R-59** hard-abort and **R-60**
|
||||
> first-boot NIC sweep, both pending). Until those ship, on any multi-NIC board confirm at the
|
||||
> installer's network step that the interface holding the lease is the one with the cable.
|
||||
3. **Bind** to the customer — **the rehearsal used the SELF-BIND path, and it is now the default for
|
||||
this runbook.** Mint the link with **"Send self-bind link"** on the customer's Setup tab; the
|
||||
this runbook.** The link is already in the customer's inbox (auto-minted at creation, A1); the
|
||||
customer opens it and completes the public two-factor `/bind/<token>` page (console pairing code
|
||||
+ retrieval passphrase). The 2026-07-18 run bound in **60 s from mint to consume, with zero failed
|
||||
attempts**, and the credential was delivered **26 s later** with no operator action.
|
||||
attempts**, and the credential was delivered **26 s later** with no operator action. **Second
|
||||
datapoint, virgin hardware:** the 2026-07-21 HP t740 onboarding walked the full chain — register
|
||||
→ pairing banner → unclaimed list → self-bind (code + passphrase) → day-0 → dashboard — on
|
||||
hardware the system had never seen (`CONTEXT.md` 2026-07-21 entry, `operations/nodes.md`).
|
||||
|
||||
> **Send the self-bind link BEFORE the customer sees the console.** The console banner tells them
|
||||
> to use „az e-mailben kapott link" — if the link has not been minted yet, the box is instructing
|
||||
> the customer to look for something that does not exist. In the rehearsal the box sat in pairing
|
||||
> mode for **~11.7 minutes** waiting on the operator, printing failures the whole time (R-33).
|
||||
> Mint it during step A1 if you can; the token is valid for **7 days**, so there is no reason to
|
||||
> wait. Auto-minting on customer create/RESET is R-36.
|
||||
> **The self-bind link exists BEFORE the customer sees the console.** The console banner tells
|
||||
> them to use „az e-mailben kapott link" — since hub v0.67.0 (R-36) the link is auto-minted at
|
||||
> customer creation and RESET completion, so the banner is true by construction (in the
|
||||
> pre-auto-mint rehearsal the box sat in pairing mode for **~11.7 minutes** waiting on the
|
||||
> operator, printing failures the whole time — R-33). Verify the mint in A1; the token is valid
|
||||
> for **7 days**.
|
||||
|
||||
Operator-bind on the Hosts page remains the fallback when the customer is not present or the
|
||||
token has expired.
|
||||
@@ -79,10 +108,10 @@
|
||||
| Drive enrol: destructive-confirm → usable (incl. mkfs) | **~30 s** |
|
||||
| Escrow ceremony started → offsite runs enabled | **12 s** |
|
||||
|
||||
Note the **PBS-DR descriptor** leg specifically: in the rehearsal it reported `converged` while
|
||||
the credential behind it was already stale, leaving the DR tier authenticating `401` with nothing
|
||||
surfacing the fault (**R-39**). Until that is fixed, treat "descriptor provisioned" as unproven
|
||||
and check `pvesm status` on the box during step E1.
|
||||
The **PBS-DR descriptor** leg no longer needs special distrust: the DR tier now self-detects a
|
||||
dead credential (a loud `auth_failed`) and self-heals via damped re-issue, and the hub gauge is
|
||||
trustworthy (**R-39 CLOSED, proven live 2026-07-21** — hub 0.68.1 + agent 0.91.2, the 13 s
|
||||
re-issue chain `applied → auth_failed → applied`; `backlog/ROADMAP.md` R-39).
|
||||
|
||||
## C. Customer zero script (the customer drives, operator narrates — ~30 min)
|
||||
|
||||
@@ -91,8 +120,9 @@ This is the friction lens: the customer's hands on the keyboard, the operator's
|
||||
1. **Claim:** open the claim e-mail → claim the dashboard → set the dashboard password.
|
||||
(R-4 gate: the e-mail must land in inbox, not spam. **gmail PROVEN 2026-07-18** — the rehearsal's
|
||||
claim mail was the first sent under the tightened DMARC `p=quarantine` and landed in the **Inbox**.
|
||||
**freemail.hu is still untested** — if your tester is on freemail.hu, treat step C1 as unproven and
|
||||
have them check spam before you conclude the mail was lost.)
|
||||
**freemail.hu PROVEN 2026-07-21** — an operator test-send was received at a freemail.hu address;
|
||||
R-4 is COMPLETE (`backlog/ROADMAP.md` R-4). Keep the habit anyway: if a mail seems missing, have
|
||||
them check spam before you conclude it was lost.)
|
||||
2. **Escrow ceremony:** run the recovery-code wizard → the awaiting card
|
||||
(„megerősítésre vár, legfeljebb 15 perc") → flips on ACK. The code goes on paper, into the
|
||||
envelope, into the customer's drawer — **state plainly: without this code, a dead box means
|
||||
@@ -101,19 +131,17 @@ This is the friction lens: the customer's hands on the keyboard, the operator's
|
||||
the destructive-confirm). Expect **~30 s** from the confirm to a usable drive — the wizard runs
|
||||
mkfs detached and polls, so a slow format is not a hang.
|
||||
|
||||
4. **Offsite:** enable „Távoli mentés". Two operator behaviours here are **workarounds for known
|
||||
defects** — follow them until the items ship, and do not improvise:
|
||||
4. **Offsite:** enable „Távoli mentés". One operator behaviour here is a **workaround for a known
|
||||
defect** — follow it until the item ships, and do not improvise:
|
||||
|
||||
> **Click Save exactly ONCE, then wait, then verify.** Provisioning runs synchronously inside the
|
||||
> request, so on a slow Hetzner leg the request can return an nginx **504 while succeeding
|
||||
> server-side**. A second click races the first attempt. Wait, then confirm on the hub's
|
||||
> Offsite → Restic tab rather than believing the browser. (**R-31**)
|
||||
> Offsite → Restic tab rather than believing the browser. (**R-31**, still open)
|
||||
|
||||
> **Re-onboarding an existing customer? Check offsite provisioning by hand.** After a RESET +
|
||||
> re-enroll the hub can sit in "enabled but unprovisioned" and **say nothing** — provisioning is
|
||||
> *Save*-triggered, and the re-enroll auto-re-issue deliberately skips unprovisioned targets. So
|
||||
> an offsite target that looks configured may never have been created. Open the offsite settings
|
||||
> and press Save once to force provisioning, then verify. (**R-36**)
|
||||
The old re-onboarding manual-Save check is gone: since hub v0.67.0 (R-36 SHIPPED,
|
||||
`backlog/ROADMAP.md`) an enabled-but-unprovisioned offsite target raises an **amber banner on
|
||||
the customer page** naming the state and the fix — trust the banner.
|
||||
|
||||
> **Expect the first run to be clean on a NEW customer.** If an **orphan card** appears
|
||||
> („a távoli mentési tároló elárvult"), the repo is not new — that is the R-32 case, which fires
|
||||
@@ -147,18 +175,17 @@ This is the friction lens: the customer's hands on the keyboard, the operator's
|
||||
- Sign the tester agreement. Leave the one-pager with them.
|
||||
- Operator record: board model + loader, drive serials, claim date, escrow ACK timestamp,
|
||||
friction notes.
|
||||
- If this relationship ever ends, the levers are **RULED and shipped** (R-25b, hub v0.69.0/v0.70.0,
|
||||
TASK-I): full teardown = the guided **DELETE cascade**; identity-preserving re-onboarding =
|
||||
**RESET** (the Friday flow). Which lever and why: **§G below**.
|
||||
|
||||
## E. Operator post-checks (same day, remote)
|
||||
|
||||
1. Hub: customer row green; Offsite → Restic tab shows the first bytes within a report cycle;
|
||||
PBS-DR gauge nominal (first PBS run on its own schedule — note, don't wait).
|
||||
|
||||
> **Verify the DR tier on the box, not from the hub (R-39, interim).** The agent can report the
|
||||
> PBS descriptor `converged` while the credential behind it is stale, and its own verify loop
|
||||
> cannot read the credential to notice. Until R-39 ships, SSH to the host and run
|
||||
> `pvesm status` — the `felhom-pbs` line must be **active**. A `401 Unauthorized` / `inactive`
|
||||
> there means the DR tier is dead despite everything upstream looking green, and it will stay
|
||||
> dead silently until someone needs a restore.
|
||||
PBS-DR gauge nominal (first PBS run on its own schedule — note, don't wait). The gauge is
|
||||
trustworthy: since R-39 closed (hub 0.68.1 + agent 0.91.2, proven live 2026-07-21 —
|
||||
`backlog/ROADMAP.md` R-39), a dead DR credential surfaces as a loud `auth_failed` and self-heals
|
||||
via damped re-issue; no manual `pvesm status` check is needed.
|
||||
2. Floor convergence: controller at current, agent at MinAgent+.
|
||||
3. Route friction notes: UX items → ROADMAP; one-pager corrections → R-11; runbook corrections →
|
||||
this doc.
|
||||
@@ -172,6 +199,7 @@ This is the friction lens: the customer's hands on the keyboard, the operator's
|
||||
- Post-rehearsal checklist item: rebuild the golden to the current controller before the FIRST
|
||||
real tester (fresh boxes shouldn't sit in a stale window longer than needed), and re-verify the
|
||||
Day-0 manifest after every controller release during alpha.
|
||||
|
||||
## G. Offboarding — which lever to pull (hub v0.69.0, R-25b)
|
||||
|
||||
Two levers, and they are not interchangeable:
|
||||
Reference in New Issue
Block a user