hub (unreleased): the record of an off-site account outlives an untick; RESET and delete take the account down (R-1375, D71); runbook: a household or a tester leaves (D74)
gates / gates (push) Successful in 2m0s
gates / gates (push) Successful in 2m0s
An untick alone takes nothing down (pinned first). Tests run the real save and RESET over the repository's fake provider. Not deployed. Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
1 parent
186f3fca58
commit
e5c643d73d
11 files changed
+563
-2
No files matched your search
@@ -402,6 +402,41 @@ attempts, 7 s apart, a log line each) in the provision and the F-14 re-issue pat
|
||||
the operator-only `pbsdr_provision_failed` („… save the customer config to retry"). Live 2026-10-26 on hub rc.1: a fresh
|
||||
Tester4 install provisioned the tier on the first try.
|
||||
|
||||
### 14.1 The off-site account when its tier is switched off [FACT — built, not released: hub `main` after v0.181.0, 2026-11-14; D71, `09` §3 decision 291, R-1375]
|
||||
|
||||
The first off-site tier (the app copies) is a sub-account at the storage provider. The household's config form
|
||||
rebuilds the whole config at every save, so until this change an UNTICK dropped the `offsite` block and the hub forgot
|
||||
the account: a later RESET or delete skipped it, and the account stayed at the provider — paid for, with the
|
||||
household's copies in it, and no record anywhere (endings EH06 and EH01).
|
||||
|
||||
- **An untick takes nothing down.** No provider call, no purge. This is the wrong case the tests pin first
|
||||
(`TestD71_UntickKeepsTheAccountAndItsRecord_ResetTakesItDown`).
|
||||
- **The hub keeps the record.** `offsite.KeepAccountRecord` runs at every save: a tier that was ticked AND provisioned
|
||||
and is now unticked leaves `offsite_account` in the config (type, user, host, the time of the untick); later saves
|
||||
carry it. It is not an `offsite` block, so the box is handed the tier as off. A tier that was ticked and never
|
||||
provisioned made no account and leaves no record.
|
||||
- **RESET and delete take the account down when a record exists** — ticked or not (`web.offsiteChoice` reads the
|
||||
tick OR the record; both previews name the account). The folder is purged first, then the account is deleted, as
|
||||
before (R-32); the record goes with the account.
|
||||
- **Ticking again** takes the same account back into use (the provider is asked by the household's label) and the
|
||||
record is gone: the descriptor is the record again.
|
||||
- **The config form says it:** „The off-site tier is OFF, and this household's account still exists at the provider”.
|
||||
|
||||
**An account that lost its record before this release** (unticked, then saved, before the fix): the hub cannot know
|
||||
it. One act finds them — at the provider's page, read each sub-account's label `felhom-customer=<id>`; for each one
|
||||
whose household has the tier unticked in the hub, tick the tier again and save (the same account comes back, and its
|
||||
record with it), then untick if it shall stay off. For a household that is gone: remove the account by hand, by its
|
||||
label (`audits/RETIRE-peti-2026-09-25.md` A3). `runbooks/household-leaves.md`, „The untick trap”.
|
||||
|
||||
**The second off-site tier** (the whole-box copies on ep0) never had this fault: switching it off keeps its place and
|
||||
its descriptor (`enabled:false`, coordinates kept — `TestPBSDR_DisableKeepsTenancy`), and RESET and delete take it
|
||||
down by the customer id, ticked or not.
|
||||
|
||||
**Not proven:** on a deployed hub, or against the real provider. The tests run the real save, the real RESET and the
|
||||
repository's fake provider (`hetznerapi.Fake`); 9 mutations seen red, one of them only after a missing test case was
|
||||
added (`audits/night-build-2026-11-14/B/redproof-hub-offsite-record.txt`).
|
||||
|
||||
|
||||
## 15. Customer e-mails — what the hub writes to a household [hub v0.118.0, R-558]
|
||||
|
||||
**The section this document did not have.** The hub composes every sentence a household reads before
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
== MUTATION: THE WRONG CASE: an untick takes the account down
|
||||
RED on the mutant, GREEN on the real file
|
||||
--- FAIL: TestD71_UntickKeepsTheAccountAndItsRecord_ResetTakesItDown (0.05s)
|
||||
d71_offsite_account_record_test.go:81: AN UNTICK TOOK SOMETHING DOWN: 0 account(s) left (want 1), 1 purge call(s) (want 0)
|
||||
== MUTATION: the record is not kept at the save (the old behaviour: the hub forgets the account)
|
||||
RED on the mutant, GREEN on the real file
|
||||
--- FAIL: TestD71_UntickKeepsTheAccountAndItsRecord_ResetTakesItDown (0.05s)
|
||||
d71_offsite_account_record_test.go:89: the account's record did not survive the untick and two saves: <nil> in {"logging":{"level":"debug"}}
|
||||
|
||||
2 mutations, 0 not a red-proof
|
||||
== MUTATION: RESET reads only the tick (an unticked account is skipped)
|
||||
RED on the mutant, GREEN on the real file
|
||||
--- FAIL: TestD71_UntickKeepsTheAccountAndItsRecord_ResetTakesItDown (0.05s)
|
||||
d71_offsite_account_record_test.go:107: the reset preview does not name the kept account u629193-sub1000: {"claim_present":false,"customer_id":"acme","dr_recipe_present":false,"escrow_ack_required":false,"host_count":0,"offsite_en
|
||||
d71_offsite_account_record_test.go:107: the delete preview does not name the kept account u629193-sub1000: {"claim_present":false,"cloudflare_manual":["the Cloudflare tunnel that serves acme.hu","the DNS records of acme.hu (the ap
|
||||
d71_offsite_account_record_test.go:123: after the RESET: 1 account(s) left (want 0), 0 purge(s) (want 1) — the unticked account was skipped
|
||||
|
||||
1 mutations, 0 not a red-proof
|
||||
== MUTATION: the record is written once and lost at the next save
|
||||
RED on the mutant, GREEN on the real file
|
||||
--- FAIL: TestD71_UntickKeepsTheAccountAndItsRecord_ResetTakesItDown (0.04s)
|
||||
d71_offsite_account_record_test.go:89: the account's record did not survive the untick and two saves: <nil> in {"logging":{"level":"debug"}}
|
||||
== MUTATION: a ticked tier keeps the old untick record
|
||||
*** NOT A RED-PROOF *** (mutant rc=0 built=True real rc=0)
|
||||
== MUTATION: a tier that was never provisioned gets a record
|
||||
RED on the mutant, GREEN on the real file
|
||||
--- FAIL: TestKeepAccountRecord_FourCases (0.00s)
|
||||
account_record_test.go:38: ticked, never provisioned, then unticked: no account was made, no record: a record &{Type:shared User: Host: UntickedAt:2026-11-14T20:00:00Z}, want none ({"git":{"username":"x"},"offsite_account":{"type"
|
||||
== MUTATION: the record outlives the account after a RESET
|
||||
RED on the mutant, GREEN on the real file
|
||||
--- FAIL: TestD71_UntickKeepsTheAccountAndItsRecord_ResetTakesItDown (0.05s)
|
||||
d71_offsite_account_record_test.go:131: the record outlived the account: {"logging":{"level":"debug"},"offsite_account":{"type":"shared","user":"u629193-sub1000","host":"u629193-sub1000.your-storagebox.de","unticked_at":"2026-11-1
|
||||
== MUTATION: the kept record is written as an offsite block (the box would read it)
|
||||
RED on the mutant, GREEN on the real file
|
||||
--- FAIL: TestD71_UntickKeepsTheAccountAndItsRecord_ResetTakesItDown (0.05s)
|
||||
d71_offsite_account_record_test.go:89: the account's record did not survive the untick and two saves: <nil> in {"logging":{"level":"debug"}}
|
||||
|
||||
5 mutations, 1 not a red-proof
|
||||
== MUTATION: a ticked tier keeps the old untick record (done again: the first run of this mutation was GREEN — the table had no case for it; the case was added)
|
||||
RED on the mutant, GREEN on the real file
|
||||
--- FAIL: TestKeepAccountRecord_FourCases (0.00s)
|
||||
account_record_test.go:40: a config that carries BOTH a ticked tier and a record: the record goes (the descriptor is the record): a record &{Type:shared User:u1-sub3 Host: UntickedAt:}, want none ({"offsite":{"enabled":true,"type"
|
||||
|
||||
1 mutations, 0 not a red-proof
|
||||
@@ -354,7 +354,7 @@ stopping line that lies.
|
||||
| **R-1293** | Hub & operator | P4 | **Once a year the hub's 02:00 database copy never reaches ep0, and the 20:00 copy is pushed twice instead.** Measured 2026-10-26 during R-1183's read (`audits/night-2026-10-25/r1183-measurement-2026-10-26.txt`): on 2026-10-25 (the clocks go back) 02:00–03:00 happens twice. The hub's `nextDailyRun` uses `time.Date`, which picks the SECOND 02:00 (CET, 01:00Z — measured with Go). DooPlex's `felhom-hub-db-backup.timer` fired at the FIRST 02:30 (CEST, 00:31Z) and pushed `hub-20261024T180000Z.db`, 391 min old — under the 7 h limit, so no refusal and no alarm; the 08:30 push took the 08:00 copy. Effect: for one morning a year ep0's newest hub copy is 12 h old, not 6 h; nothing is lost on the hub. The spring night (2027-03-28, 02:00–03:00 does not exist) is not measured. **2026-11-11:** the Clock list leans on this row (`documentation/audits/clock-list-2026-11-11/TABLE.md` CH23-6). | OPEN | — | pick one: the push refuses (and says so) a newest snapshot older than the slot it serves, or the hub and the timer agree on one 02:00; then read the 2027-03-28 night (DUE-CHECKS) | CC |
|
||||
| **R-1345** | Hub & operator | P4 | **An access token's scopes cannot be changed after it is made — widening or narrowing one means a new token (and a new secret).** The operator, 2026-10-29: to give CC `hosts:read` + `hosts:write` for the live proofs he had to make `CC_HOSTS` (now in `sec`) beside `cc-releases-2`. Wanted: edit a token's scopes in the session (an audit line per change), so a scope can also be taken away after a session without retiring the secret. | OPEN | — | design + build the scope edit (session only), red-first test that an unticked scope is refused at once | CC |
|
||||
| **R-1353** | Hub & operator | P4 | **Statements still over 1 s, named by the new slow-statement line (hub 0.178.0) in its first hour:** a background deadline/restore-test check reading days of full host-report JSON (`GetHostReportsSince`, `store.go`; 1.5–9.2 s cold, from `monitor/deadline.go` and `monitor/restoretest.go` — no page waits on it); the start-up prune of `events` and `app_log_issues` (1.1 s and 1.8 s; no index on `created_at` / `last_seen`); the customer page's newest-report read (`GetCustomer`, 1.3 s cold in the pod probe). With the 1 GiB limit most of this should come from the cache once warm. Evidence: the hub log 2026-10-30 09:22–10:05 UTC; `audits/release-2026-10-30/p2-r1346/`. | OPEN **2026-11-02 read (`audits/security-2026-11-02/part4/hub-slow-lines-2026-11-01-02.txt`; the log covers only the 31 h since DooPlex restarted on 2026-11-01 16:11 CET, the lines from 10-30 are gone with the old pod):** 35 lines, 15 statements. What repeats: `VACUUM INTO` 5× (avg 110 s, the 6-hourly snapshot, expected); `GetHostReportsSince` 9× (avg 7.9 s, max 26.6 s); `GetCustomer`'s newest report 3× (1.1 s). Once, at the cold start: the prune of `events` (10.4 s) and `app_log_issues` (21 s, again 04:30), the OS-release and host lists (2–22 s). No fix before the full week (11-06). **2026-11-10 read (the hub's start after a deploy, `audits/release-space-fixes-2026-11-10/hub/10-watch.txt`):** 11 slow lines in the first 2 minutes of the new pod, the longest 19.1 s (`GetHostReportsSince`), then the OS-report reads (3–4 s) and the two prunes (2.7 s, 4.2 s); none in the 28 minutes after. The 13:02 UTC snapshot's `VACUUM INTO` took 127.8 s. The same statements as the 11-02 read; no new one. | — | read a week of slow-statement lines; index or narrow what still repeats (the history read needs only a few fields, not the whole JSON) | CC |
|
||||
| **R-1375** | Hub & operator | P2 | **The outside part of three endings is done by hand or not at all — the worst: an off-site account can stay at the provider with no record in the hub.** Found 2026-11-10 (the endings list, `documentation/audits/endings-list-2026-11-10/TABLE.md` endings EH06, EH01, EA21 and the ending "a tester ends"; read in source and documents; no account was called). (1) **Unticking the off-site tier** of a household takes nothing down and leaves no reminder (`hub/internal/web/configs.go:1737-1743`). A later RESET takes the account down only when the tier is ticked (`hub/internal/web/customer_reset.go:203-245`), so the sub-account and its data stay, are paid for, and the hub holds no record that they exist. (2) **Switching the second off-site tier off** takes nothing down: storage entry, token and copies stay; the source calls the teardown 'a deliberate future op' (agent `internal/pbsdr/manager.go:274`). (3) **A tester ends**: no handler and no runbook; done by hand once (2026-09-25); no list says what to collect. **Members (answers marked N):** EH06-4, EH01-4, EA21-4, and 'a tester ends' in the table's head. Related: R-810 (a household leaves), R-32. | OPEN — filed 2026-11-10 (endings list) **2026-11-10 15:19 — RULED (D71 = yes, D74 = yes; `09` §3 decisions 291, 294).** Not built. | — | Hub (D71): keep the account record after an untick; RESET and delete take the account down, ticked or not. Docs (D74): the runbook 'a household or a tester leaves', from the five code doors, in a later documents-only session. Parts (2) and (3) of this row have no ruling of their own. | CC |
|
||||
| **R-1375** | Hub & operator | P2 | **The outside part of three endings is done by hand or not at all — the worst: an off-site account can stay at the provider with no record in the hub.** Found 2026-11-10 (the endings list, `documentation/audits/endings-list-2026-11-10/TABLE.md` endings EH06, EH01, EA21 and the ending "a tester ends"; read in source and documents; no account was called). (1) **Unticking the off-site tier** of a household takes nothing down and leaves no reminder (`hub/internal/web/configs.go:1737-1743`). A later RESET takes the account down only when the tier is ticked (`hub/internal/web/customer_reset.go:203-245`), so the sub-account and its data stay, are paid for, and the hub holds no record that they exist. (2) **Switching the second off-site tier off** takes nothing down: storage entry, token and copies stay; the source calls the teardown 'a deliberate future op' (agent `internal/pbsdr/manager.go:274`). (3) **A tester ends**: no handler and no runbook; done by hand once (2026-09-25); no list says what to collect. **Members (answers marked N):** EH06-4, EH01-4, EA21-4, and 'a tester ends' in the table's head. Related: R-810 (a household leaves), R-32. | NARROWED — part 1 (D71) is built, not released. **2026-11-14 (night build):** the hub keeps the record of an off-site account after the tier is unticked; RESET and delete take the account down when a record exists; an untick alone takes nothing down (pinned by a test); the config form says that the account still exists (`05` §14.1). The runbook „a household or a tester leaves” is written (D74, `runbooks/household-leaves.md`). **For the operator, once, after the release:** at the provider's page read each sub-account's label `felhom-customer=<id>`; for a household whose tier is unticked in the hub, tick it again and save — the account's record comes back. LEFT: part (2) has no fault (the second tier keeps its record, read in source and its test); part (3) is the runbook; a household mail at leaving stays R-810. OPEN — filed 2026-11-10 (endings list) **2026-11-10 15:19 — RULED (D71 = yes, D74 = yes; `09` §3 decisions 291, 294).** Not built. | — | Hub (D71): keep the account record after an untick; RESET and delete take the account down, ticked or not. Docs (D74): the runbook 'a household or a tester leaves', from the five code doors, in a later documents-only session. Parts (2) and (3) of this row have no ruling of their own. | CC |
|
||||
| **R-1376** | Hub & operator | P3 | **Records of a thing that is gone stay in six places.** Found 2026-11-10 (the endings list, `documentation/audits/endings-list-2026-11-10/TABLE.md` endings EH00, EH02, EH21, EA00, EA11, EA27; read in source, `documentation/audits/endings-list-2026-11-10/evidence/hub-delete-statements.txt`; no hub database was read). (1) **Hub:** thirteen tables have no delete statement anywhere. Deleting a box leaves its update settings, its update reports and its kernel notices; deleting a household leaves its off-site key rows, off-site window rows and abandon requests. Most are small; the update reports of each box grow with time. (2) **Host uninstall** leaves the mount units and host mounts of household drives (said when R-179 closed; no open row carried it). (3) **A drive wiped by a signed job** keeps a record and a mount unit that name the old filesystem (agent `internal/signedjobs/wipe.go:61-110`). (4) **The token of a destroyed guest** stays valid until a new guest takes its number: the token store has no remove (agent `internal/localapi/tokenstore.go:127-210`). **Members (answers marked N):** EH00-4, EH02-4, EH21-4, EA00-4, EA11-4, EA27-4. | OPEN — filed 2026-11-10 (endings list) **2026-11-10 15:19 — RULED (D73 = yes, with one exception; `09` §3 decision 293).** Not built. | — | Hub (D73): add the tables with no delete to the two purges and to the daily clean-up; the delete journal (`host_deletions`, `customer_resets`) stays on purpose, and the privacy text says so. Agent: remove a guest's token when the guest is destroyed. Parts (2) and (3) have no ruling of their own. | CC |
|
||||
| **R-1386** | Hub & operator | P3 | **Timed jobs with no time limit and no measured duration (25 answers).** Found 2026-11-11 (the Clock list, class W4, `documentation/audits/clock-list-2026-11-11/TABLE.md`; read in source; the hub was not measured, by ruling). The hub's 60-second sweep runs its checks one after the other with no limit for each: one slow check delays all the others, the liveness check too (CH19-3, CH19-8, CH01-3, CH08-3, CH08-8, CH10-3, CH13-8, CH18-3, CH06-8, CH02-3); the 07:10 audit has 10 minutes, measured with three boxes only (CH11-3, CH11-8). On the box: the interval jobs have no limit of their own (CC11-3, CC12-3, CC13-3, CC14-3, CC15-3, CC16-3, CC20-3, CA06-3, CA10-3, CA10-8, CA13-8, CA15-3, CA19-3). Every measured duration is from a small test store. | OPEN — filed 2026-11-11 (clock list) | — | Hub: log each sweep step's duration and alarm when one sweep takes longer than its interval; then read a week of numbers and set limits. Box: the same for the controller's interval jobs. | CC |
|
||||
| **R-1393** | Hub & operator | P4 | **For about 8 minutes after a hub restart its database statements take 1 to 32 seconds; the System page took 12.7 s and a box's enrolment 11.4 s.** Measured 2026-11-14 at the 0.181.0 deploy (`audits/release-1-dates-2026-11-14/hub/05-watch.txt`): 24 `slow statement` lines from 09:58 to 10:06 UTC, none in the 20 minutes after. The database file is 678 MB; the slow ones read `host_reports`, `os_reports` and `events` (whole report JSON per row). One 32-second statement ran while CC copied the database out of the pod for a read (CC's own load). No report was lost that CC could see; not measured: whether a box's report timed out in that window. | OPEN — filed 2026-11-14 | — | measure a restart with no copy running; then decide: an index or a shorter keep on `host_reports`/`os_reports`, or accept | CC |
|
||||
|
||||
@@ -0,0 +1,131 @@
|
||||
# A household or a tester leaves — the runbook
|
||||
|
||||
> **Status: written 2026-11-14 from source and from one real use; NOT run as a whole.** Operator ruling D74
|
||||
> (`09-update-architecture.md` §3 decision 294, R-1375). Each step says **PROVEN** (with where) or **NOT PROVEN**.
|
||||
> A step that is NOT PROVEN is not a wrong step: nobody has seen it work. Read its line before you press.
|
||||
> Source: the endings list (`audits/endings-list-2026-11-10/TABLE.md`, endings EH00 to EH06) and the one real
|
||||
> retirement (`audits/RETIRE-peti-2026-09-25.md`). The design of what the hub removes is `05-hub-architecture.md` §14.
|
||||
|
||||
## What this is, and what it is not
|
||||
|
||||
A household leaves Felhom, or a tester stops testing. You remove what Felhom holds of them: in the hub, at the
|
||||
off-site provider, on ep0, and at Cloudflare. **There is no single button.** The hub has five doors; three are
|
||||
buttons, one is a tick box, one is by hand.
|
||||
|
||||
It is NOT for a household that starts again on a new machine. For that, use RESET only (door 2) and stop.
|
||||
|
||||
## Before you start — four questions
|
||||
|
||||
1. **Does the household want its data?** The box holds the data. The hub holds none of it. Tell the household to
|
||||
copy what it wants off the box (the file manager, or each app's own export) BEFORE you start. After door 3 the
|
||||
off-site copies are gone and there is no undo. **NOT PROVEN:** no text for the household exists (capability E16;
|
||||
R-810). Write to them by hand.
|
||||
2. **Is the box still on?** An ONLINE box refuses doors 1 and 3. Ask the household to switch the box off, or wait
|
||||
until the hub shows it as down (a silent box can be deleted after 6 minutes). PROVEN in tests
|
||||
(`hub/internal/web/hosts.go`, the 409 gate; `customer_delete_test.go` GatesFailClosed).
|
||||
3. **Was the off-site tier ever ticked for this household, and is it ticked now?** Read the household's config page.
|
||||
If it was ticked once and is unticked now, see „The untick trap” below before door 3.
|
||||
4. **Who is it?** Write the customer id and the host id on paper. You type both.
|
||||
|
||||
## The five doors, in the order you use them
|
||||
|
||||
| # | Door | Where | What it ends | Ending |
|
||||
|---|---|---|---|---|
|
||||
| 1 | Delete the box | hub → Hosts → the box → Delete | the box's record | EH00 |
|
||||
| 2 | RESET the household | hub → the household → Reset | everything the box made, at the hub and outside; the household's record stays | EH01 |
|
||||
| 3 | Delete the household | hub → the household → Delete customer… | doors 1 and 2 for every box, then the record itself | EH02 |
|
||||
| 4 | Untick a tier | hub → the household → Edit | the tier is no longer offered to the box; NOTHING is removed | EH06 |
|
||||
| 5 | Cloudflare, by hand | Cloudflare's dashboard | the tunnel, the DNS records, the API token | EH05 |
|
||||
|
||||
Doors 2 and 3 open two more by themselves: the off-site account (EH03) and the copies on ep0 (EH04).
|
||||
|
||||
**For a household that leaves: door 3, then door 5.** Door 3 does doors 1 and 2 for you.
|
||||
**For a tester who stops: the same.** There is no other path for a tester (the endings list, „a tester ends”).
|
||||
|
||||
## The steps
|
||||
|
||||
### Step 1 — the household copies its data off the box. NOT PROVEN (no product path; by hand).
|
||||
|
||||
### Step 2 — the box is off, or silent for 6 minutes.
|
||||
PROVEN in tests: an online box refuses (`TestCustomerDelete…GatesFailClosed`). NOT PROVEN on a real leaving
|
||||
household.
|
||||
|
||||
### Step 3 — read the preview. hub → the household → Delete customer…
|
||||
The dialog shows what will go: the boxes, the off-site account, the residue count, and the Cloudflare objects you
|
||||
must remove by hand. **Read the off-site line.** It must name an account when the household ever had the off-site
|
||||
tier. If it says „none” and you know there was one: STOP, go to „The untick trap”.
|
||||
PROVEN: the preview was read at the one real use (`RETIRE-peti-2026-09-25.md` A3 step 3: 0 hosts, the sub-account,
|
||||
residue 1,519). The Cloudflare line: PROVEN in a test only (`TestR688_PreviewNamesCloudflareByHand`).
|
||||
|
||||
### Step 4 — tick the three boxes, type the customer id, press.
|
||||
The hub then runs, in this order: the journal entry → every box deleted (door 1) → the off-site account taken down
|
||||
→ the copies on ep0 taken down → the claim reset → the hub's rows purged → the record and ALL key packages purged.
|
||||
- Door 1 for each box: PROVEN in tests (`TestDeleteHost_CascadeAndIsolation`); a real box delete is routine
|
||||
(Tester4, 2026-11-11 and 11-14, by the operator).
|
||||
- The off-site account: **PROVEN once on a real account** (`RETIRE-peti-2026-09-25.md`: sub-account 269130 removed,
|
||||
the pool's other accounts and their sizes unchanged). That account held no backup. **NOT PROVEN: an account that
|
||||
holds a real backup.** The hub empties the folder through the account's own login first, then deletes the account;
|
||||
if the emptying fails the account is KEPT and the step fails (R-32) — then press again.
|
||||
- The copies on ep0: **NOT PROVEN on real copies** (at the one real use there was nothing on ep0: `existed=false`).
|
||||
Tests only (`customer_delete_test.go`). The second copy of ep0 on DooPlex is NOT removed by this and was not
|
||||
measured (`runbooks/ep0-datastore-copy.md`).
|
||||
- The purge of the hub's rows: PROVEN once (482 reports, 1,036 telemetry rows, 1 log tail gone; journal #20).
|
||||
**Known gap, R-1376:** rows in 13 tables stay (the box's update settings and reports, its kernel notices, the
|
||||
off-site key and window rows, the abandon requests). Ruled to be removed (D73, `09` §3 decision 293); read the
|
||||
row for whether that is released — until then they stay.
|
||||
|
||||
### Step 5 — if a leg fails: press again.
|
||||
The dialog offers Resume. A leg that is done is not done again. Nothing is half forgotten: the journal names the leg.
|
||||
PROVEN in tests (`…ResumesAfterExternalFailure`). NOT PROVEN live (`_recovery-inventory-2026-07-28.md`: „the
|
||||
end-to-end live leg is not run”).
|
||||
|
||||
### Step 6 — check the provider by hand. 5 minutes.
|
||||
At the off-site provider's page: the household's sub-account is gone; the others are there; the pool's size did not
|
||||
jump. PROVEN as a check (it is what `RETIRE-peti-2026-09-25.md` A4 did). **Always do it when the journal says
|
||||
`skipped_no_config`** — the hub could not know the account then.
|
||||
|
||||
### Step 7 — door 5, Cloudflare, by hand. NOT PROVEN (never done; no code does it; the operator accepted that, R-688).
|
||||
From the dialog's list: 1. delete the tunnel; 2. delete the DNS records that point at it; 3. delete the household's
|
||||
API token; 4. if the zone was added only for this household, remove the zone or leave it to the household (it is
|
||||
THEIR domain — ask them). If you do nothing: the names point at a tunnel that no longer runs; a visitor sees
|
||||
Cloudflare's error page. No data is exposed.
|
||||
|
||||
### Step 8 — what stays, on purpose. Do not „clean” these.
|
||||
- The audit trail in the hub: the delete and reset journals, the host-deletion records, the events and the
|
||||
notification log. The last two go by themselves after one year (R-901, PROVEN in a test).
|
||||
- The hub's own database snapshots: they hold the household's rows for up to 8 weeks on ep0. That is the only way
|
||||
back, and it is a retention fact you must be able to state to the household.
|
||||
- The household's box and its drives: theirs. Felhom removes nothing on the box. If the household wants the box
|
||||
wiped, that is a new install by them, or the box's own „factory” path — NOT PROVEN as part of leaving.
|
||||
|
||||
### Step 9 — a tester only: what to collect. NOT PROVEN (no list existed; this is the first).
|
||||
Before step 4, ask the tester for: what did not work (in their words); whether they want to come back later (then
|
||||
use RESET, door 2, and keep the record); the box's hub page saved as a file if a finding is open in the register.
|
||||
After step 7: remove the tester from `runbooks/target-selection.md` (as Peti's entry was), and from `STATUS.md` if
|
||||
named there.
|
||||
|
||||
## The untick trap (door 4) — read this if the off-site tier was unticked
|
||||
|
||||
Unticking the off-site tier removes nothing at the provider. **Until the fix is released, it also made the hub
|
||||
forget the account:** a later RESET or delete then skipped it, and the account stayed, paid for, with the
|
||||
household's copies in it, and no record in the hub (R-1375, endings EH06 and EH01).
|
||||
|
||||
- **Fix built, not released (hub `main`, 2026-11-14, D71):** the hub keeps the account's record after an untick, and
|
||||
RESET and delete take the account down when a record exists.
|
||||
- **Until that release, and for any household unticked before it:** before door 3, tick the off-site tier again and
|
||||
save (this is ending EH06's own undo and gives the record back), THEN do door 3. NOT PROVEN on a real household.
|
||||
- **If the record is already lost** (unticked, then RESET or deleted, before the fix): the account has the label
|
||||
`felhom-customer=<customer id>` at the provider. Find it by the label, never by its number, and remove it by hand
|
||||
as `RETIRE-peti-2026-09-25.md` A3 did. PROVEN once (that record).
|
||||
|
||||
## The second off-site tier (the whole-box copies on ep0)
|
||||
|
||||
Switching it off keeps the household's place and copies on ep0, on purpose (PROVEN in a test,
|
||||
`TestPBSDR_DisableKeepsTenancy`). The hub finds them by the customer id, so door 2 and door 3 take them down whether
|
||||
the tier is ticked or not. Nothing to do by hand here.
|
||||
|
||||
## What is not in this runbook, and why
|
||||
|
||||
- A mail to the household that says what was removed. It does not exist (R-810). Write by hand.
|
||||
- A refund or an invoice. Money is the operator's.
|
||||
- A wipe of the household's box. Not Felhom's to do.
|
||||
Reference in new issue
Block a user