hub v0.104.0: the guest network gets a reader (R-319), and the hub half of the naming (R-295)
gates / gates (push) Successful in 14s
gates / gates (push) Successful in 14s
Four paper debts and one fact given a reader. Hub-only — nothing to bake. A4 — the entry about "the tester's machine" named a risk correctly and labelled it in a way that invited deleting it. Established from the hub's own store: `peti-felhom` is a REAL machine (482 reports, 2026-02-27 → 2026-07-15, a named person's own box) and the 3.6 GB with no key and no backup is real. `david` → `tester-1` is a DIFFERENT record with no host, no escrow and no report, ever — deleted 07:55:49 and re-created 07:56:47 this morning. The prompt's premise conflated the two; the register now says which is which. A1 — R-312/R-313/R-303 recorded as DECIDED with their re-open triggers, and moved out of STATUS's "Waiting on you", which is now empty. A3 — day0-install §C.1 said pushing the installer publishes it. It has not since R-110. Corrected, with the two manifest pins named and an outside-verification command; the one copy that repeated it (a dated audit, true when written) carries a superseded note. A5 — standing rule 5: evidence comes off the machine at the end of the phase that produced it, before any revert. Earned twice in three days on the same box at the same point (R-320). Four homes, plus what to do when it is already gone. R-295 hub half — „Beállító kód" everywhere; „Visszaállító kód" retired. New `reenroll` mail kind so the mail names the page a REBUILT box actually shows („A szerver beállítása"), not the „Elfelejtett jelszó" page it has no login screen to reach. Naming only; the acceptance pin proves the secret is untouched. R-319 — the hub models `guest_net` after 23 days of receiving and discarding it. The signal is `heals_last_hour`, not `state`: a guest the watchdog keeps repairing reads healthy between repairs. `heal_succeeded` decoded too (R-260's lesson). Unknown is never drawn as healthy — three absences, three sentences. No alarm, deliberately. Three red-proofs, mutations asserted applied. Wire-gate checked tags 182 → 190. B1 — the operator's 2026-08-12 dispositions were NOT in the register; they are now. Third allowlist kind for the five ruled "no reader wanted"; `reporting_disabled` reclassified redundant. 8 read · 5 deliberately unread · 1 redundant · 6 still owed. Also filed: R-321 (a deliberately-silent box still alarms stale/down — the checker is age-only, and decoding the flag would not have fixed it), R-322 (the claim guard has never scanned the hub; a hand scan returns zero, so it is a scope gap, not a defect).
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# STATUS — what works, what's broken, what's next
|
||||
|
||||
**Updated 2026-08-12 (night — the door, part one).**
|
||||
**Updated 2026-08-13 (evening — the small debts, and one fact given a reader).**
|
||||
|
||||
> **A view, not a source.** `documentation/backlog/OPEN-ITEMS.md` is the authority; this page restates
|
||||
> part of it in plain words, and **nothing may exist only here**. **Items, not paragraphs. One screen.**
|
||||
@@ -8,44 +8,25 @@
|
||||
|
||||
## Waiting on you
|
||||
|
||||
*Three decisions. Each says what it would cost to leave alone, because two of them quietly choose an
|
||||
outcome if you do not answer. The register row is the detail, not the decision.*
|
||||
*Nothing. All three questions that stood here were answered on 12–13 August and have moved to
|
||||
**Decided** below. A decided question left in the deciding list is how a person loses track of what is
|
||||
actually waiting.*
|
||||
|
||||
### Should a customer be able to get their old backups back themselves, or is that a phone call?
|
||||
## Decided — and what would reopen each
|
||||
|
||||
Since Tuesday a machine recognises an older recovery code and says so honestly, but it cannot hand the
|
||||
files over — it tells the person to write to us, and we can do it by hand.
|
||||
*A decision with no trigger becomes a permanent silence, so each one names what would make us look
|
||||
again.*
|
||||
|
||||
- **Build it:** the restore code has to accept a second location and password instead of only its own.
|
||||
Contained — three functions and a screen — plus one genuine design question: what a customer sees
|
||||
when they have several old sets and must pick one.
|
||||
- **Leave it:** nothing breaks. Every customer in this position becomes a support conversation, and we
|
||||
keep a promise we can only keep manually.
|
||||
|
||||
**If you do nothing:** the honest message stays and the work never gets scheduled. Nobody is blocked;
|
||||
this is the one decision here with no deadline of any kind. *(register: R-312)*
|
||||
|
||||
### The old copy on the demo machine cannot be opened by anyone. Keep paying to store it, or delete it?
|
||||
|
||||
You told me to keep it, and I did. Then I found out what it is: 36 backups and a single key, and that
|
||||
key was destroyed by the bug we fixed on 4 August. **No recovery code in existence opens it.**
|
||||
|
||||
- **Keep it:** pennies of storage, and it stays as the one physical example of what that bug cost.
|
||||
- **Delete it:** irreversible, and the example goes with it.
|
||||
|
||||
**If you do nothing:** it stays forever and stops being a decision — which is how five scratch
|
||||
customers accumulated. Nobody is blocked. *(register: R-313)*
|
||||
|
||||
### When a machine is in two kinds of trouble at once, should it say both things?
|
||||
|
||||
A machine can count down to deleting its old backups while also reporting that it cannot open its new
|
||||
ones. Both cards are true; together they are bewildering.
|
||||
|
||||
- **Leave it:** two true statements, confusing side by side. Nobody has been hurt by it.
|
||||
- **Hide the older card during a countdown:** tidier, and it risks hiding a real second failure — which
|
||||
is why I have not done it.
|
||||
|
||||
**If you do nothing:** both keep showing. Ranked low on purpose. *(register: R-303)*
|
||||
- **Getting old backups back yourself: NOT BUILT, deliberately.** A customer in that position is a
|
||||
support conversation, and we can do it by hand. **Reopens if:** a real customer actually asks —
|
||||
one request from a person who is not us. *(register: R-312)*
|
||||
- **The unopenable old copy on `demo-felhom`: KEPT, as a test fixture.** Not for sentiment: it is the
|
||||
only state in existence where a set-aside store is present and cannot be opened, which is the case
|
||||
any future handling of lost backups has to face honestly. **Delete it when:** that work ships, or is
|
||||
abandoned. Until then it is a fixture, not an accumulation. *(register: R-313)*
|
||||
- **A machine in two kinds of trouble says both things: LEFT AS IT IS.** Its real-world likelihood is
|
||||
unknown, and hiding one card risks hiding a real second failure. **Reopens if:** it is observed
|
||||
happening outside a constructed test. *(register: R-303)*
|
||||
|
||||
## What works
|
||||
|
||||
@@ -53,53 +34,50 @@ Both demo machines are home, healthy and reporting on the approved pair — **co
|
||||
0.129.0**, delivered by the floor rather than by hand. Off-site is credentialed on `demo-hp` and its
|
||||
repository still opens with the machine's own key. `drill-r50` is reverted to `virgin`, powered off.
|
||||
|
||||
**What the fleet actually is, because two summaries have now been misread:** the hub holds **five
|
||||
customer records and three machines**. The machines are `demo-felhom` and `demo-hp` (both ours, both
|
||||
disposable) and `drill-r50` (a nested drill VM on DooPlex, reverted and powered off). **`peti-felhom`
|
||||
is a real machine we have not heard from since 15 July** and has no host record. **`tester-1` is a
|
||||
record with no machine** — created 13 August, no host, no backups, nothing to lose.
|
||||
|
||||
## Shipped
|
||||
|
||||
- **The drive can be re-attached after a reinstall** (R-280). The restore page said *"this is two
|
||||
clicks"* over an empty list; it was zero clicks and needed an internal path no customer could produce.
|
||||
- **The orphan card stops promising** that set-aside off-site copies can be reopened — twice over
|
||||
(R-294, then **R-299**, which was the same claim in the plural, in the *always-visible* half, missed
|
||||
because the guard matched one inflection of a Hungarian verb).
|
||||
- **The countdown banner stops promising retrieval it cannot see is still true** (R-302). The promise
|
||||
is now conditional on the hub still holding the package it held when the customer decided — pinned
|
||||
then, compared now. A sweep found the same claim in five places; a fourth was fixed with it and a
|
||||
fifth deliberately left, because it is true where it renders.
|
||||
- **One name per secret, box side** (R-295): the dashboard code is „Beállító kód" everywhere;
|
||||
„Visszaállító kód" is retired. It collided with the escrow „Helyreállítási kód" and cost a real code.
|
||||
- **The removal now genuinely reverses the installation** (R-316, `installer-v1.28.0` published). The
|
||||
second reinstall used to hit our own leftover; it was watched failing on the cycle that actually
|
||||
fails, then watched passing.
|
||||
- **A correct recovery code is no longer called wrong** (R-311, three components). If a customer types
|
||||
the code for an older set of backups, the machine now checks the packages we kept, recognises it, and
|
||||
says so: *your code is correct, it belongs to an earlier package, we kept it, your current backups are
|
||||
fine, write to us*. It deliberately promises no restore, because there is no button yet.
|
||||
- **The countdown on `demo-felhom` is cancelled** on your ruling (R-307). Nothing was deleted; the
|
||||
24 August deadline is gone. See R-313 for what that copy turns out to be.
|
||||
- **Vouched and delivered 2026-08-12**: golden 0.214.0, agent 0.129.0, floor 0.214.0 — both machines
|
||||
took it themselves. The version guard was watched working on the way: `demo-hp` was **held** back
|
||||
while its agent was older, and updated 6 seconds after the agent caught up. That guard exists because
|
||||
a machine once ran ahead of its agent and a customer was told a correct code was wrong; first sighting.
|
||||
- **Both installer fixes are now PUBLISHED** as `installer-v1.27.0` (R-297 + R-300). Each fault was
|
||||
watched happening first, on a machine reset to factory state: the old installer really did build a
|
||||
machine on a base image from July, and our own uninstall really did block our own next install.
|
||||
- **The drive can be re-attached after a reinstall** (R-280), and **the orphan card and the countdown
|
||||
banner stop promising retrieval they cannot see is still true** (R-294, R-299, R-302).
|
||||
- **One name per secret — now both halves** (R-295). The dashboard code is „Beállító kód" everywhere,
|
||||
on the machine *and* in the hub's emails; „Visszaállító kód" is retired. It collided with the escrow
|
||||
„Helyreállítási kód" and cost a real code.
|
||||
- **The hub can see whether a machine's guest still has working networking** (R-319, first reader built
|
||||
against R-264). A machine quietly repairing its own network over and over is now visible instead of
|
||||
being a green tick; a machine that does not report it is drawn as unknown, never as healthy.
|
||||
- **The countdown on `demo-felhom` is cancelled** on your ruling (R-307). Nothing was deleted.
|
||||
|
||||
## Broken, or knowingly incomplete
|
||||
|
||||
- **The tester's machine has no recovery route at all** — see the `PETI` row. Its host record was
|
||||
deleted on 15 July; there is no key, no off-site copy and no local backup. **If that drive fails,
|
||||
everything on it is lost.** First act of the visit: copy the ~3.6 GB off before anything is
|
||||
reinstalled — it is currently the only copy in existence. Whether it stays parked is your call and is
|
||||
deliberately left open.
|
||||
- **Kept backups can be opened — but still only by us** (R-304 partly closed, R-312 open). The machine
|
||||
now recognises an older code and says so plainly instead of hedging. What it still cannot do is hand
|
||||
the customer their old files: that needs the restore code to accept a second location, which is real
|
||||
work rather than wiring. Today the honest answer is "your code is right, write to us" — and we can.
|
||||
- **The dnsmasq fix helps a machine once** (R-305). On a machine that never had Felhom it works. On the
|
||||
second reinstall the leftover comes back, because the package is never removed — so the machine looks,
|
||||
to our own installer, as if the household had installed it. Watched happening the same afternoon.
|
||||
- **The hub half of the naming is undone** (R-295 PARTIAL): the emails still use the retired name and
|
||||
send people to a page a rebuilt machine does not show.
|
||||
- **Peti's machine has no recovery route at all** — see the `PETI` row. **This is a real machine
|
||||
belonging to a real person**, not one of ours and not a record: it reported to the hub for four and a
|
||||
half months and has been silent since 15 July, when its host record was deleted. There is no key, no
|
||||
off-site copy and no local backup. **If that drive fails, everything on it is lost.** First act of the
|
||||
visit: copy the ~3.6 GB off before anything is reinstalled — it is currently the only copy in
|
||||
existence. Whether it stays parked is your call and is deliberately left open.
|
||||
- **Kept backups can be opened — but still only by us** (R-304 partly closed, R-312 decided-not-built).
|
||||
Today the honest answer is "your code is right, write to us" — and we can.
|
||||
- **The agent picks dnsmasq by looking at a file another package owns** (R-317). The box installs fine;
|
||||
only LAN name resolution goes missing, and quietly. One line, deliberately not taken tonight.
|
||||
- **The storage page has its own separate reason for showing an empty list** (R-298), untouched.
|
||||
- **Three more facts the machines send still have no reader** (R-264): a staged-but-unapplied agent
|
||||
update, how deep a restore test actually went, and the two backup-integrity timestamps. Five others
|
||||
are now recorded as deliberately unread, which is honest rather than fixed.
|
||||
|
||||
## Working on next
|
||||
|
||||
R-312's shape (the button, or deliberately no button); then R-305, because the tester's second
|
||||
reinstall still hits the dnsmasq wall; then the hub naming; then the 2026-08-09 batch
|
||||
(R-279 … R-292), still untriaged against everything since.
|
||||
The three remaining R-264 readers, now that one has been built and we know what one costs; then R-317
|
||||
(one line in the agent); then the 2026-08-09 batch (R-279 … R-292), still untriaged against everything
|
||||
since.
|
||||
|
||||
@@ -476,7 +476,14 @@ Report MUST include:
|
||||
6. **Deployed versions** + `docker ps` / agent-service / hub-pod verification output.
|
||||
7. **NOT yet live-validated — awaiting supervised [Bx]:** explicit list (the real
|
||||
put-data → operate → integrity flow).
|
||||
8. **Teardown evidence — all three §13 layers**, if the run provisioned anything: the machine deleted,
|
||||
8. **Evidence copied off BEFORE each revert** — for every phase that ran on a machine, the logs were
|
||||
pulled to the evidence directory **at the end of that phase**, before any revert, snapshot restore
|
||||
or teardown, **including the intermediate ones**. *The intermediate revert is the one that gets
|
||||
forgotten: two sessions lost a phase's logs to a mid-run revert to `virgin` on 2026-08-12 and
|
||||
2026-08-13 — same machine, same point, three days apart (R-320).* **If a phase's evidence is
|
||||
already gone, the report says so plainly and the finding is REPRODUCED independently** — that is the
|
||||
expectation, not an improvisation. A quotation read live and no longer re-readable is named as such.
|
||||
9. **Teardown evidence — all three §13 layers**, if the run provisioned anything: the machine deleted,
|
||||
`pvesm status` before/after with the space returned, and the **hub-side record's disposition named**
|
||||
(deleted / retained-with-reason / gate-blocked-with-the-command). A run that provisioned nothing says
|
||||
so. "Teardown clean" without layer 3 is not a report — it is the `sess-c` failure.
|
||||
|
||||
@@ -126,6 +126,7 @@
|
||||
| Multiple household users / per-person accounts | — | **MISSING** | — | Single dashboard password; acceptable for alpha → R-15 |
|
||||
| WireGuard base infra always-on; OOB operator access (felhom-sshd, /32 peer) | agent v0.72, hub v0.35 | **IMPLEMENTED** | `SPIKE-oob-wg-operator-peer-2026-07-05`, `SPIKE-felhom-sshd-2026-07-05` | Mutual-repair desired-state arc not built → R-13. **CHECKED 2026-08-08 (R-260) and this row was NOT claiming something untrue** — it claims the capability is implemented, never that it is monitored, so no correction was owed. What WAS untrue is narrower and sat one layer down: **the hub's own OOB health check could not see whether the operator's key was installed.** `HostOOBRow` mirrored five of the agent's eight OOB fields, so `operator_key_configured` — emitted every heartbeat since agent v0.72.0, i.e. from this row's own vintage — was discarded by `encoding/json` on arrival, and `oobDegraded` returned `ok` for a box with felhom-sshd active, reachable, a valid config, a configured peer and **no operator key at all**. `operator_peer_configured`, which it did read, only says the peer IP is in desired-state — that OOB is MEANT to work, not that entry is possible. Fixed hub v0.99.0; the missing key now degrades and the alert NAMES it; a stanza too old to carry the field is reported distinctly and is never a silent ok. Pinned end-to-end from raw report JSON by `TestHostOOB_MissingOperatorKey_EndToEnd` and `TestHostOOB_NoKeyField_IsNotSilentlyOK_EndToEnd` |
|
||||
| The operator can see WHERE a managed host is — its LAN address and its WireGuard address, on the host page | agent **v0.119.0**, hub **v0.85.0** | **PROVEN-LIVE** (2026-07-31) | `audits/host-addresses-visible-2026-07-31.md` | Before this the LAN IP was **not reportable at all** — `HostMetrics` carried no address of any kind — and the WG IP existed only in `/offsite`'s peer table keyed by pubkey (peer→host, never host→peer). New wire field `addresses[]`, one row per (interface, address); `IsGlobalUnicast()` is the whole filter, chosen by MEASURING both demo boxes, and it needs no veth/fwbr denylist because that plumbing carries no IP. Rendered live on both 0.119.0 hosts matching their `ip addr` ground truth exactly. **Two honesty properties carry the risk and are both red-proofed:** WireGuard shows the hub ALLOCATION and whether the box CONFIRMS holding it (allocation alone cannot tell a live tunnel from a peer never applied), and an agent below 0.119.0 renders **UNKNOWN, never "no addresses"** — proven live on `drill-r50-0a4f9a` (0.113.0). **Not covered:** a two-LAN-bridge box and a real WG drift, neither of which exists to observe |
|
||||
| The operator can see whether a managed host's **guests still have working networking** — and **how often the watchdog had to repair them** | agent **v0.92.0** (emitter, 2026-07-21), hub **v0.104.0** (reader, 2026-08-13) | **IMPLEMENTED** | `backlog/OPEN-ITEMS.md` R-319; `hub/internal/web/hosts_guestnet_test.go` (7 tests, fixtures copied verbatim from `demo-felhom-8363b5`'s live `host_reports` row) | The agent emitted `guest_net` on every heartbeat for **twenty-three days** while the string occurred **nowhere** in `felhom.eu/hub/` — stored as raw text in `report_json`, read by nothing (R-260/R-264, the first of that census's readers to be built). **The fact that carries the risk is `heals_last_hour`, not `state`:** a guest the watchdog keeps repairing is healthy at every instant anyone looks, so rendering the state alone would give it a green tick — the failed-disk-drawn-as-a-healthy-empty-disk shape. `heal_succeeded` is decoded beside it, because six FAILED repairs is a guest that is down while six successful ones is a nuisance. **Unknown is never drawn as healthy:** three absences, three sentences (agent < 0.92.0; a capable agent that sent nothing; a guest whose own state the watchdog did not assert), and a malformed stanza degrades to unknown without a 500. **Three red-proofs, each mutation asserted applied by grep before its run**, including the one that matters — removing the unknown branches and watching a silent machine render as healthy. **Positive control that it is WIRED and not merely written: the wire-contract gate's checked-tag count rose 182 → 190** as the eight `guest_net` allowlist entries were deleted (an allowlisted tag is skipped, so leaving them would have meant these fields were never checked) | **IMPLEMENTED, not PROVEN-LIVE, and the distinction is the honest half.** Every scenario is proven against the real wire in tests, and the healthy case renders correctly for the live fleet — but **no machine has ever been observed with a climbing repair count on this card**, because neither demo box has needed a repair since the watchdog shipped. The signal this card exists for has therefore never been seen firing on hardware. It moves to PROVEN-LIVE the first time a real repair count is watched appearing. **No alarm was added, deliberately** (R-319): the incident behind this was about nobody being able to SEE the condition, and a new email on a fleet of two demo machines is untested noise — revisit when a third machine exists or when a count is seen climbing |
|
||||
| Break-glass management-plane recovery | agent v0.71, hub v0.84 | **IMPLEMENTED** | `runbooks/break-glass.md` | hub v0.84.0 adds an **operator-SESSION** retrieval path (host page → Console access → Reveal; `POST /hosts/{id}/reveal-recovery-credential`, CSRF-gated, writes a customer-visible `recovery_credential_revealed` event) beside the pre-existing **global-key** one (`GET /api/v1/admin/hosts/{id}/recovery-credential`), which is untouched and stays the route for when the hub UI itself is down. **The credential half is now PROVEN (2026-07-31):** the vaulted `demo-hp-bb76ea` password was verified against the box's own `/etc/shadow` hash AND minted a real PVE ticket — `POST /api2/json/access/ticket` → **HTTP 200, `root@pam`, 367-char ticket**, the exact API the login form submits to. Still IMPLEMENTED rather than PROVEN-LIVE overall, because the path has not been exercised on a REAL lockout (SSH was available throughout). Discovered during that check: the card's Copy button shipped `disabled` until a Reveal and silently no-opped, leaving ANOTHER host's password in the clipboard — fixed in hub v0.86.0. The vaulted secret is plaintext at rest → **R-133** |
|
||||
|
||||
## F. Notifications & monitoring
|
||||
|
||||
@@ -0,0 +1,170 @@
|
||||
# REPORT — The removal that works once, and a status page that says what it is asking for (2026-08-13)
|
||||
|
||||
**Shipped:** `installer-v1.28.0`, published and verified against the live URL.
|
||||
**Register:** R-305 CLOSED (by R-316), **R-316 / R-317 / R-318 opened**; ceiling R-315 → **R-318**.
|
||||
**Venue:** `drill-r50` only. Neither demo box reinstalled. `peti-felhom` not contacted. Nothing deleted.
|
||||
|
||||
---
|
||||
|
||||
## 1. Cycle 3, before and after
|
||||
|
||||
**Before — on the PUBLISHED v1.27.0, from `virgin`, exit 1:**
|
||||
|
||||
```
|
||||
[ERROR] a resolver is already bound to :53 on this host:
|
||||
udp UNCONN 0 0 0.0.0.0:53 … users:(("dnsmasq",pid=7076,fd=4)) …
|
||||
[ERROR] a resolver is already bound to :53 on this host — Felhom needs the guest reachable by name on your LAN.
|
||||
Stop or reconfigure that resolver, OR point your LAN DNS at the guest's address, then re-run.
|
||||
(Felhom does NOT touch DNS services on a host it does not own — this is a refusal, not a change.)
|
||||
THIS LOOKS LIKE OURS. A previous Felhom install leaves the dnsmasq PACKAGE installed and its unit
|
||||
enabled (only our config snippet is removed), and unconstrained it binds 0.0.0.0:53 — which is what
|
||||
this gate is seeing. If this host had no dnsmasq before Felhom, clear it with:
|
||||
systemctl disable --now dnsmasq
|
||||
Then re-run this installer. If dnsmasq is YOURS, leave it and use one of the two routes above.
|
||||
[ERROR] PRE-FLIGHT FAIL (exit 1) — fix the finding above and re-run
|
||||
```
|
||||
|
||||
**After — v1.28.0, three fresh cycles from `virgin`, exit 0:**
|
||||
|
||||
```
|
||||
:53 now: 0
|
||||
CYCLE 3 rc=0
|
||||
[INFO] host DNS (:53): free
|
||||
[OK] pre-flight passed
|
||||
[OK] PRE-FLIGHT PASS (mode=byo) — no state written, no install step executed
|
||||
```
|
||||
|
||||
The three cycles were run on bytes **sha256-identical** to what the live URL now serves
|
||||
(`cca1dedd9b7c152c…`, compared three ways: live URL, tested file, repo main).
|
||||
|
||||
## 2. The mechanism, at `file:line`
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| ownership **recorded** | `felhom-host-install.sh:1718` — preflight, `dpkg-query -W -f='${Status}' dnsmasq \| grep -q "install ok installed"` |
|
||||
| ownership **read** | `:1097` (`_state_get dnsmasq_preexisting`) at uninstall |
|
||||
| state file **deleted** | `:1268` — *after* the read. **The order was already correct.** |
|
||||
| Felhom's dnsmasq snippets removed | `:1162`, in the loop just above the ownership decision |
|
||||
|
||||
**Why cycle 2 concludes "pre-existing": package presence alone.** The preflight asks dpkg one question
|
||||
and nothing else — it does **not** consult the absence of a record, and it cannot, because the record
|
||||
was deleted with the state file. v1.27.0 stopped the unit and left the package, so the answer stayed
|
||||
`yes` and our own package became "the household's" one cycle later.
|
||||
|
||||
**What the first uninstall did and did not remove:** snippets — removed. Unit — stopped + disabled.
|
||||
State file (and with it the record) — removed. **Package — left.** That last one is the whole defect.
|
||||
|
||||
## 3. What v1.28.0 changes
|
||||
|
||||
When the record says we installed it, the uninstall **removes the package** as well as stopping the
|
||||
unit. Order unchanged: read the record → act → delete the state.
|
||||
|
||||
**Two packages are recorded, not one.** `dnsmasq` ships the systemd unit; **`dnsmasq-base` ships
|
||||
`/usr/sbin/dnsmasq`** (`dpkg -S`, measured on the box). Separately installable, so each is recorded at
|
||||
preflight and taken back only if we added it.
|
||||
|
||||
**Guard rails:** ownership **read, never inferred**; the dependency check is an **`apt-get -s purge`
|
||||
simulation** that proceeds only if the removal set is a subset of ours, else stop+disable **naming the
|
||||
blocking package**; never interactive; **never fatal** — a wedged apt is recorded and restated in the
|
||||
closing NOTE; and the success is **re-queried** rather than read off apt's exit code.
|
||||
|
||||
## 4. Scenarios and red-proofs
|
||||
|
||||
| Scenario | Result |
|
||||
|---|---|
|
||||
| **A** three cycles, fixed | install 3 **PASSES**, `:53` free at every uninstall |
|
||||
| **B** resolver pre-dates Felhom | **untouched** — `install ok installed`, unit active, both packages recorded `yes` |
|
||||
| **C** something depends on it | **not purged**; log named `household-dns-thing`; stop+disable; `:53` free; dependent survived |
|
||||
| **D** no ownership record | **untouched**, reason logged, exact command named |
|
||||
|
||||
**Red-proofs — mutation asserted applied before each run:**
|
||||
|
||||
| Mutation | Outcome |
|
||||
|---|---|
|
||||
| remove the purge call | cycle 2 records `yes`; **cycle 3 refuses, exit 1, in those exact words** |
|
||||
| remove the ownership check | **the household's resolver is PURGED** (`unknown ok not-installed`) |
|
||||
| infer ownership when no record | **the guess is taken** — a field box loses its own DNS |
|
||||
|
||||
**One honest note on red-proof B.** Its first run appeared to pass *for the wrong reason*: apt failed
|
||||
with `dpkg was interrupted` (a broken state my own earlier manual package juggling left), so the
|
||||
resolver survived by accident rather than by the guard. I repaired dpkg and re-ran, and it then failed
|
||||
as required. Worth stating twice over: the mutation looked like a passing guard, and **my own fix's
|
||||
apt-failure path was incidentally observed doing exactly what it should** — logging, continuing, naming
|
||||
the command.
|
||||
|
||||
## 5. The machines already in the field
|
||||
|
||||
**Measured, not reasoned.** A field box is one with the package installed and **no record** — scenario
|
||||
D. Its uninstall leaves the resolver running with the reason logged, and its next install refuses with
|
||||
the message quoted in §1.
|
||||
|
||||
**Is there an honest durable marker? No, and none can be invented.** The Felhom `/etc/dnsmasq.d/
|
||||
felhom-*.conf` snippets are deleted by the uninstall's own loop *before* the ownership decision; the
|
||||
state file carrying the record is deleted at `:1268`; nothing under `/etc/felhom*` survives.
|
||||
`/var/log/dpkg.log` does record the install and is a **timestamp** — refused by the standing rule as a
|
||||
heuristic dressed as a fact. → **R-318**
|
||||
|
||||
**The preflight message, judged as a customer would:** it states the finding, keeps its two routes and
|
||||
its written promise not to touch DNS on a host we do not own, and adds the one thing that gets a person
|
||||
moving — *"THIS LOOKS LIKE OURS"* plus one exact command. It hedges correctly (*looks like*). Its
|
||||
weakness is that it asks them *"did this host have dnsmasq before Felhom?"* — precisely the question we
|
||||
can no longer answer for them. **That is a mechanism, not a rule: the command is on the screen at the
|
||||
moment it is needed.**
|
||||
|
||||
## 6. The defect I nearly shipped
|
||||
|
||||
The agent decides whether to install dnsmasq with `os.Stat("/usr/sbin/dnsmasq")`
|
||||
(`felhom-agent/internal/lanresolver/lanresolver.go:105`) — but that path belongs to **`dnsmasq-base`**,
|
||||
while the unit comes from **`dnsmasq`**. Purging only `dnsmasq` would leave the binary, so the next
|
||||
install would skip the apt step and then fail to enable a unit that is gone — **a silent resolver where
|
||||
today there is at least a visible refusal.** That is why ownership is recorded per package.
|
||||
|
||||
It remains reachable where `dnsmasq-base` pre-dated Felhom (we correctly keep it). **Pre-existing, not
|
||||
introduced here**, filed as **R-317**, and the uninstall now says so out loud rather than leaving it to
|
||||
be found from a resolver that never came up. Fixing it is a one-line agent change, deliberately **not**
|
||||
made here to keep this session to one repo.
|
||||
|
||||
## 7. Publication
|
||||
|
||||
`installer-v1.28.0` tagged; **both** `--ref`s in `manifests/webpage.yaml` moved (sidecar 327, init 372);
|
||||
ArgoCD synced to `823cd29`; live deployment carries the new ref.
|
||||
|
||||
```
|
||||
live URL serves: SCRIPT_VERSION="1.28.0" (was 1.27.0)
|
||||
_dnsmasq_purge_owned present in the served file: 3
|
||||
```
|
||||
|
||||
**Pushing publishes nothing here** — `/scripts/` follows the tag. The runbook still says otherwise;
|
||||
**R-309 remains open and is not forgotten**, deliberately out of scope.
|
||||
|
||||
## 8. Teardown — four layers
|
||||
|
||||
| Layer | State |
|
||||
|---|---|
|
||||
| **machine** | `drill-r50`: 0 guests, no `/var/lib/felhom-install`, dnsmasq `not-installed` at the end |
|
||||
| **host** | `drill-r50` **reverted to snapshot `virgin`, powered off**; `qemu.pid` removed. DooPlex was drill host only |
|
||||
| **hub** | `drill-r50-0a4f9a` **RETAINED** — pre-existing since 2026-07-25, re-used not duplicated. **No new host or customer row was created this session.** Nothing deleted |
|
||||
| **off-site** | **Not contacted at all** — no restic, sftp or endpoint call was made. `demo-felhom`: `last_status=ok`, `escrow=escrowed`, **no abandon countdown** |
|
||||
|
||||
Both demo boxes untouched and healthy: controller **0.214.0**, agent **0.129.0**, `Up … (healthy)`.
|
||||
|
||||
## 9. Evidence, and a repeated miss
|
||||
|
||||
Logs at `documentation/audits/evidence-r316-2026-08-13/`: the fixed cycles (`F1`, `F2`, `F3`), all four
|
||||
scenarios (`B`, `C`, `D`), and every red-proof (`RPA*`, `RPB*`, `RPD*`).
|
||||
|
||||
**The Part 1 logs did not survive.** They were on the drill VM's disk and were destroyed by the revert
|
||||
to `virgin` between Part 1 and Part 2 — **the same mistake as Tuesday, in the same place.** The §1
|
||||
quotation is verbatim from the live run as read at the time, and `RPA1/RPA2/RPA3.log` are an
|
||||
independent reproduction of the identical three-cycle failure, retained. Recorded rather than glossed;
|
||||
the fix is procedural and I have now got it wrong twice.
|
||||
|
||||
## 10. Observations — noticed, not acted on
|
||||
|
||||
- **`dnsmasq-base` is flagged "automatically installed and no longer required"** after `dnsmasq` goes.
|
||||
We deliberately do not `autoremove` — that would be a blast radius nobody asked for.
|
||||
- **`apt-get -s purge` exits 0 even when it prints `E: dpkg was interrupted`.** The simulation output is
|
||||
the signal, never the exit code — which is why the guard parses `Remv`/`Purg` lines rather than
|
||||
trusting `$?`. Another entry for the exit-codes-that-lie class.
|
||||
- The drill VM's `pveam` index is stale on `virgin` and needs `pveam update` before listing templates —
|
||||
already recorded yesterday, hit again today in passing.
|
||||
@@ -183,6 +183,14 @@ no first-boot hook at all, so this question now governs **operator-built images
|
||||
`main` on a 30-second period (`documentation/.../day0-install.md:150-152`) — no release tag, no
|
||||
staging copy, no version selector. Pushing the script publishes it.
|
||||
|
||||
> **⚠ SUPERSEDED 2026-08-03, annotated 2026-08-13 (R-110, R-309). True on the day it was written;
|
||||
> false ever since.** `/scripts/` is now served from the tag `installer-v<SCRIPT_VERSION>` and
|
||||
> **pushing publishes nothing** — publication is a tag plus **two** `--ref` pins in
|
||||
> `manifests/webpage.yaml`. The dated finding is kept as written rather than rewritten, because it
|
||||
> is a record of what was true then; **the note is here because this paragraph cited
|
||||
> `day0-install.md` as its source, which is how the wrong sentence spread.** Current procedure and
|
||||
> the outside-verification command: `runbooks/day0-install.md` §C.1.
|
||||
|
||||
3. **Console display does not exist.** `felhom-host-install.sh` does not write `/etc/issue`,
|
||||
`/etc/issue.net` or any MOTD (grep: no match). PVE's own `/etc/issue` banner is what the console
|
||||
shows after install — observed in this session's screendumps as
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -26,6 +26,13 @@
|
||||
> **Point of no return:** booting the install-armed stick (the auto-installer needs no confirm).
|
||||
> Recovery unchanged: stock ISO + `felhom-host-install.sh` — and since RESET is the plan, there
|
||||
> is nothing to restore. Demo-felhom.eu is down for the window (~2 h).
|
||||
>
|
||||
> **EVIDENCE RULE — standing rule 5 (R-320), and it applies to every phase boundary below.** The logs
|
||||
> of a phase are copied OFF the machine at the end of **that** phase, before any revert, snapshot
|
||||
> restore or teardown — **including the intermediate ones, which are the ones that get forgotten.**
|
||||
> Two sessions lost a phase's logs exactly this way on 2026-08-12 and 2026-08-13, same machine, same
|
||||
> point. Make the pull the last act of the phase. **If evidence is already gone: say so plainly and
|
||||
> reproduce the finding independently** — do not let a lost log quietly become a softer claim.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -167,9 +167,44 @@ chmod +x felhom-host-install.sh
|
||||
./felhom-host-install.sh -h | head -3 # sanity: must print v1.15.0 (or newer) — DR-tier-by-default ships the full plumbing
|
||||
```
|
||||
|
||||
This URL is the website's git-sync working tree tracking `main` on a 30-second period
|
||||
(`manifests/webpage.yaml`) — it is always the current `main` script. There is no release tag, no
|
||||
staging copy and no version selector; pushing `scripts/felhom-host-install.sh` publishes it.
|
||||
**⚠ THIS PARAGRAPH USED TO SAY THE OPPOSITE OF THE TRUTH, and two sessions were misled by it before
|
||||
it was corrected on 2026-08-13 (R-309, R-110).** The old text read *"it is always the current `main`
|
||||
script … pushing `scripts/felhom-host-install.sh` publishes it."* **Pushing publishes NOTHING, and has
|
||||
not since R-110 shipped on 2026-08-03.** Believing the old sentence is dangerous in **both**
|
||||
directions: it makes an operator think a pushed fix is live when it is not, and think a pushed mistake
|
||||
is live when it is not. (Measured 2026-08-12: this URL served `1.25.0` while `main` held `1.27.0`,
|
||||
three and a half hours after the push.)
|
||||
|
||||
`manifests/webpage.yaml` runs **two** git-syncs against **different refs**: the website tracks `main`,
|
||||
and `/scripts/` is checked out from the tag **`installer-v<SCRIPT_VERSION>`**. So this URL serves the
|
||||
**tagged** installer, not `main`.
|
||||
|
||||
**What actually publishes it — three acts, and the middle one is two lines, not one:**
|
||||
|
||||
1. Bump `SCRIPT_VERSION` in `scripts/felhom-host-install.sh` (the single version source) and push to
|
||||
`main`. *Nothing is published yet.*
|
||||
2. Cut and push the tag `installer-v<new SCRIPT_VERSION>`.
|
||||
3. Move **BOTH** `--ref=installer-v…` pins in `manifests/webpage.yaml` — the git-sync **sidecar** and
|
||||
the **init container** (today lines 327 and 372) — commit, and sync ArgoCD. **Moving one pin is the
|
||||
trap**: the running pod keeps serving until it restarts, and then a fresh pod seeded by the stale
|
||||
init container serves the OLD script with no error anywhere.
|
||||
|
||||
**To roll back:** move the tag back and wait ~30 s. No ArgoCD sync, no deploy — that is the emergency
|
||||
lever; fix forward afterwards. **Do not pin the website to the tag**, or every copy edit becomes a
|
||||
release. `hostinstall_gates.py` gate 6 fails if the manifest stops naming an `installer-v…` tag or if
|
||||
the website stops tracking `main`.
|
||||
|
||||
**How to verify from OUTSIDE that the published version actually changed** — a push, a green sync and a
|
||||
correct-looking manifest are each consistent with nothing having been published, so ask the public URL
|
||||
rather than the repository:
|
||||
|
||||
```bash
|
||||
curl -fsS https://felhom.eu/scripts/felhom-host-install.sh | grep -m1 '^SCRIPT_VERSION'
|
||||
```
|
||||
|
||||
It must print the **new** version. Confirmed by this method on 2026-08-13: served `1.28.0`, `main`
|
||||
`1.28.0`, both manifest pins `installer-v1.28.0` — the three agreeing is the observation, and any one
|
||||
of them alone is not.
|
||||
|
||||
### C.2 Preview (recommended)
|
||||
|
||||
|
||||
@@ -25,6 +25,25 @@ Fences name **acts**, not machines. "Do not destroy demo-hp's `drill-r50` fixtur
|
||||
demo-hp to host a throwaway VM" are unrelated; only the first has ever been meant. Read a per-machine
|
||||
prohibition as covering the act it names and nothing more.
|
||||
|
||||
## Before you revert it — take the evidence off first
|
||||
|
||||
> **A phase's evidence is copied off the machine at the end of THAT phase, before any revert, snapshot
|
||||
> restore or teardown. Not at the end of the session.** (Standing rule 5, R-320.)
|
||||
>
|
||||
> **The intermediate revert is the one that gets forgotten.** Both losses this project has recorded
|
||||
> were the *middle* teardown, never the final one — the Phase A logs of the 2026-08-12 retained-key
|
||||
> drill and the Part 1 logs of the 2026-08-13 R-316 run, both on `drill-r50`, both destroyed by a
|
||||
> revert to `virgin` between phases, three days apart. Both times the conclusions survived only
|
||||
> because the quotations had been read live and an independent reproduction happened to exist. **That
|
||||
> is luck.**
|
||||
>
|
||||
> **The mechanism:** make the pull the last act of the phase, not a step to remember later —
|
||||
> `pct pull` / `scp` into `documentation/audits/evidence-<topic>-<date>/` on DooPlex, then revert.
|
||||
> Tier 0 machines are *disposable*, which is precisely why nothing you need may be left on one.
|
||||
>
|
||||
> **If you notice it is already gone:** say so plainly in the report and **reproduce the finding
|
||||
> independently**. That is the documented expectation, not an improvisation.
|
||||
|
||||
| Tier | Meaning | Machines |
|
||||
|---|---|---|
|
||||
| **0 — disposable. Reach here first.** | Exists to be broken; reinstalling is a routine afternoon, not an incident. **A drill that needs a victim uses one of these.** | `demo-hp` (t740), `demo-felhom` (N100) |
|
||||
|
||||
@@ -56,6 +56,15 @@ roles. **A file being open in the editor is NOT an instruction. If no task is st
|
||||
"working" and "stopped entirely".
|
||||
4. **A recommendation that is not followed gets one line saying why.** Silence reads as agreement and
|
||||
the disagreement is lost.
|
||||
5. **Evidence is copied off the machine at the end of the phase that produced it — before any revert,
|
||||
snapshot restore or teardown. Not at the end of the session.** The intermediate revert is the one
|
||||
that gets forgotten; both losses were the *middle* teardown, never the final one. **The mechanism,
|
||||
because a rule without one is a wish:** the last act of a phase that ran on a machine is `scp`/`pct
|
||||
pull` its logs to the evidence directory on DooPlex — the same act that ends the phase, not a
|
||||
separate step to remember later. **And when a session notices the evidence is already gone: say so
|
||||
plainly in the report and REPRODUCE it independently.** That is the documented expectation, not an
|
||||
improvisation to be invented under pressure — it is what both sessions did, and it is the only
|
||||
reason two sets of conclusions survived.
|
||||
|
||||
<!--
|
||||
R-96 incident record (committed 2026-07-27) — rationale, not directives.
|
||||
@@ -72,6 +81,16 @@ R-96 incident record (committed 2026-07-27) — rationale, not directives.
|
||||
4. Twice in the R-88/R-97 arc a review point was absorbed rather than argued: R-84 was folded into
|
||||
R-82 without a word, and R-97a's operator-only guard was dropped while the claim it was meant to
|
||||
enforce got committed as a comment — which is how a false guarantee shipped and survived a release.
|
||||
5. Twice in three days, on the SAME machine and at the SAME point. 2026-08-12, the retained-key drill:
|
||||
the Phase A logs lived on `drill-r50`'s disk and were destroyed by the revert to `virgin` between
|
||||
Phase A and Phase B (`audits/DRILL-retained-key-2026-08-12.md` §11.5). 2026-08-13, R-316: the Part 1
|
||||
logs, same disk, same revert, between Part 1 and Part 2 (`audits/REPORT-r316-installer-v1.28.0-2026-08-13.md`
|
||||
§9 — "the same mistake as Tuesday, in the same place"; preserved out of `REPORT.md`, which is
|
||||
overwritten every session). Both times the golden-bake runbook's existing "scp the log OUT first"
|
||||
was applied to the FINAL teardown and not the intermediate one. Both times the conclusions survived
|
||||
only because the quotations had been read live and an independent reproduction happened to exist —
|
||||
that is luck, and the second occurrence is what makes it procedural rather than another apology.
|
||||
Filed as R-320.
|
||||
-->
|
||||
|
||||
## Shared conventions
|
||||
|
||||
@@ -1,3 +1,110 @@
|
||||
## v0.104.0 — the hub can see whether a guest's networking works, and one name per secret (2026-08-13, R-319 + R-295 hub half)
|
||||
|
||||
Two things, both hub-only. **No wire change, no agent change, no controller change, nothing to bake.**
|
||||
|
||||
### R-319 — the guest-network watchdog finally has a reader
|
||||
|
||||
The agent has reported `guest_net` on every heartbeat since **v0.92.0** (R-54, 2026-07-21). The string
|
||||
`guest_net` occurred **nowhere** in this repository: the stanza was stored as raw text inside
|
||||
`report_json` and read by nothing — no check, no alarm, no screen. This is the first of R-264's
|
||||
twenty-one unconsumed facts to be given a reader, and it was chosen because it has a live incident
|
||||
behind it: a killed `dhclient` in a guest took a tunnel down for 1 h 15 m with nobody told
|
||||
(`audits/INCIDENT-guest-dhclient-killed-2026-07-20.md`).
|
||||
|
||||
**Established before anything was written.** The facts arrive and are persisted:
|
||||
`demo-felhom-8363b5`'s newest row carries `guest_net.checked_at` plus per-guest
|
||||
`vmid/state/mode/ip/has_route/dhclient_alive/checked_at/message`, and the agent's wire type
|
||||
(`felhom-agent/internal/hub/report.go:139-160`) additionally carries `healed`, `heal_succeeded`,
|
||||
`last_heal_at`, `heals_last_hour`, `damped` — absent from the live rows only because they are
|
||||
`omitempty` on a box that has never needed a repair.
|
||||
|
||||
**THE SIGNAL IS THE REPAIR COUNT, NOT THE STATE.** A guest whose network the watchdog keeps repairing
|
||||
is healthy at every instant anyone looks and is nevertheless failing. Rendering `state` alone would
|
||||
give that machine a green tick — the exact shape of the defect that drew a failed disk as a healthy
|
||||
empty disk. `heals_last_hour` is therefore surfaced *beside* the state, not behind it, and drives the
|
||||
card's summary badge.
|
||||
|
||||
`heal_succeeded` is decoded too, deliberately. **R-260 is this project's standing warning that a
|
||||
decoder three fields short of the agent dropped the one that decided the question the checker
|
||||
existed to answer.** Six failed repairs is a guest that is down; six successful ones is a nuisance.
|
||||
|
||||
**An unknown is never drawn as healthy.** Three absences are kept apart, each with its own sentence,
|
||||
following the companion-flag convention (`hostNetworkView` is the sibling):
|
||||
|
||||
- the agent predates v0.92.0 — absence is expected, and still tells us nothing;
|
||||
- a capable agent sent no stanza — the watchdog is default-on, so this means it was switched off;
|
||||
- a guest whose own `state` is empty or unrecognised — the state switch is an ALLOW-LIST, so adding a
|
||||
state to the agent can never silently paint it green here.
|
||||
|
||||
A malformed stanza degrades to unknown and **must not 500 the page** — one box's bad field cannot
|
||||
break the host page for the rest of the fleet.
|
||||
|
||||
**No alarm was added, and that is a judgement rather than an omission.** The incident behind this was
|
||||
about nobody being able to *see* the condition, not about nobody being paged; and a new alarm on a
|
||||
fleet of two demo machines is untested noise on a dispatcher whose severity contract is exact-match
|
||||
lowercase. Revisit when a third machine exists, or when a repair count is observed climbing on real
|
||||
hardware — the visible count is what will supply that evidence.
|
||||
|
||||
**Red-proofs, each mutation asserted applied by grep before its run and reverted after:**
|
||||
|
||||
1. the unknown branches replaced by the healthy badge → **both** scenario-C sub-cases go red, a silent
|
||||
machine seen rendering as healthy. *(the one that matters)*
|
||||
2. `RepairCount: 0` in the decoder → B goes red on *"a guest repaired 6 times in an hour is reported
|
||||
as fine"*.
|
||||
3. `Degraded()` forced true → A goes red on *"a machine that is fine is being alarmed on"* — the guard
|
||||
is reachable in both directions, not just the alarming one.
|
||||
|
||||
**Positive control that the wiring is real and not merely written:** `wire_contract_gate.py`'s
|
||||
checked-tag count rose **182 → 190** and its skipped count fell **88 → 80**, because the eight
|
||||
`guest_net` allowlist entries were REMOVED. An allowlisted tag is *skipped*, so leaving them would
|
||||
have meant the new reader's own fields were never checked for reachability at all.
|
||||
|
||||
### R-295 hub half — one name per secret, and the mail names the page the machine shows
|
||||
|
||||
The box side shipped 2026-08-10; the hub side was dropped twice and was the last place the retired
|
||||
name survived — in the **mails**, the one surface a customer reads *before* they see any screen.
|
||||
|
||||
- the three-word dashboard code is **„Beállító kód"** everywhere: the mail bodies, the mail subjects,
|
||||
the operator button, and the lockout notice;
|
||||
- the ten-word escrow code stays **„Helyreállítási kód"**, and no dashboard-code mail names it —
|
||||
naming both secrets in one message is how a customer comes to believe they are the same thing;
|
||||
- **„Visszaállító kód" is retired.**
|
||||
|
||||
**New email kind `reenroll`, because the mail must name the page the machine is actually showing.**
|
||||
`ReissueForReenroll` sent the *reset* mail, which directs the customer to an „Elfelejtett jelszó"
|
||||
page — but a rebuilt box has no password, so the controller renders „A szerver beállítása"
|
||||
(`web/claim.go:279`, `reset := s.authEnabled()`) and serves **no login page at all**. The route named
|
||||
was not on their screen. Only the hub can tell the two situations apart, because it is the hub that
|
||||
chose which call site fired. Same secret, same name, different sentence — which is exactly what the
|
||||
ruling asks for. The re-enrol mail deliberately says **nothing** about apps or backups: a clean-slate
|
||||
reinstall is precisely where such a reassurance could be false.
|
||||
|
||||
**This is naming, not function.** No acceptance logic moved.
|
||||
`TestReenrollSplit_ChangesTheMailNotTheSecret` pins it: the generation still advances, the stored
|
||||
bcrypt hash still verifies the emailed code, no plaintext is persisted. On the box side the companion
|
||||
pin is the controller's `TestResetCode_StillAcceptedOnTheSetupPage`.
|
||||
|
||||
**Two red-proofs:** routing re-enrolment back to `EmailReset` reddens both the new pin and the
|
||||
pre-existing `TestReissueForReenroll`; putting „Visszaállító kód" back in the reset body reddens
|
||||
`TestFormatClaimEmail_OneNamePerSecret`.
|
||||
|
||||
**The claim guard has never scanned the hub.** `retrieval_promise_gate.py` lives in
|
||||
`felhom-controller/controller/scripts/` and its declared surfaces are that repo's templates plus one
|
||||
Go handler file. A scan of `felhom.eu/hub/` with the gate's own four stems returns **zero** — so
|
||||
nothing was hiding, and the new strings carry no stem and need no registration. That the gate's scope
|
||||
excludes a customer-facing surface is recorded as R-322 rather than fixed here.
|
||||
|
||||
### Also
|
||||
|
||||
- `wire_contract_gate.py` grows a **third** entry kind — `not consumed, deliberately` — carrying the
|
||||
operator's 2026-08-12 rulings with their date. Five facts move to it; `reporting_disabled` moves to
|
||||
`redundant`. **Those rulings were not in the register before today**, which is why R-264 now records
|
||||
them. Where the twenty stand: **8 read · 5 deliberately unread · 1 redundant · 6 still owed**.
|
||||
- **R-321 filed, not fixed:** the staleness checker is age-only, so a box on which reporting is
|
||||
deliberately switched off still alarms `node_stale` then `node_down`. Decoding `reporting_disabled`
|
||||
would have felt like progress and left the alarm firing; the fix belongs in the checker, which
|
||||
already holds the `health.status = "disabled"` it needs.
|
||||
|
||||
## v0.103.0 — a host can read the packages we kept for it (2026-08-12, R-311)
|
||||
|
||||
**`ListSupersededEscrow` had zero production callers for nineteen days.** It is the only reader of a
|
||||
|
||||
@@ -29,8 +29,21 @@ type EmailKind string
|
||||
|
||||
const (
|
||||
EmailClaim EmailKind = "claim" // first setup: "Elindult a Felhom szervered"
|
||||
EmailReset EmailKind = "reset" // forgotten password
|
||||
EmailReset EmailKind = "reset" // forgotten password — the box HAS a password
|
||||
EmailClaimed EmailKind = "claimed" // confirmation after a successful claim (carries no code)
|
||||
// EmailReenroll — the box was wiped and re-enrolled, so the fresh controller has NO password
|
||||
// while the hub-side claim is still set (ReissueForReenroll).
|
||||
//
|
||||
// R-295 (hub half, 2026-08-13): THIS EXISTS BECAUSE THE MAIL MUST NAME THE PAGE THE MACHINE IS
|
||||
// ACTUALLY SHOWING. Both situations deliver the same secret and it keeps the same name — the
|
||||
// three-word „Beállító kód" — but they do NOT show the same screen, and only the hub can tell
|
||||
// them apart, because it is the hub that chose which call site fired. A rebuilt box renders
|
||||
// „A szerver beállítása" (controller `web/claim.go:279`: `reset := s.authEnabled()`, and a fresh
|
||||
// controller has no password), and it serves no login page — so it has no „Elfelejtett jelszó"
|
||||
// link at all. Sending a re-enrolled customer to that page names a route that is not on their
|
||||
// screen. Splitting the KIND rather than the NAME is what the ruling asks for: one secret in two
|
||||
// situations keeps its name, and the sentence around it changes.
|
||||
EmailReenroll EmailKind = "reenroll"
|
||||
)
|
||||
|
||||
// Mailer delivers a claim-arc email. The code is passed through and MUST NOT be persisted or
|
||||
@@ -178,7 +191,8 @@ func (e *Engine) ReissueForReenroll(cc *store.CustomerConfig) (gen int, reissued
|
||||
if cs == nil || !cs.Claimed() {
|
||||
return 0, false, nil // unclaimed → first-provision path; nothing to re-issue
|
||||
}
|
||||
gen, err = e.rotateAndSend(cc, EmailReset)
|
||||
// EmailReenroll, not EmailReset: same secret, same name, different screen — see the constant.
|
||||
gen, err = e.rotateAndSend(cc, EmailReenroll)
|
||||
if err != nil {
|
||||
return gen, true, err // reissued=true so the caller records the attempt even on email failure
|
||||
}
|
||||
|
||||
@@ -166,8 +166,14 @@ func TestReissueForReenroll(t *testing.T) {
|
||||
if !cs.Claimed() {
|
||||
t.Fatal("re-issue must NEVER un-claim (reset rides rotation)")
|
||||
}
|
||||
if len(m.sends) != sendsBefore+1 || !strings.HasPrefix(m.sends[len(m.sends)-1], "reset:") {
|
||||
t.Fatalf("claimed re-enroll must send exactly one RESET email, got %v", m.sends)
|
||||
// R-295 hub half (2026-08-13): this used to assert "reset:". The KIND was split — same
|
||||
// secret, same name („Beállító kód"), different SENTENCE — because a rebuilt box has no
|
||||
// password, so it shows „A szerver beállítása" and serves no login page, and the reset mail
|
||||
// sent the customer to an „Elfelejtett jelszó" page that is not on their screen. The
|
||||
// assertion is deliberately kept STRICT rather than loosened to "either kind": routing a
|
||||
// re-enrolment back down the reset copy is exactly the regression worth failing on.
|
||||
if len(m.sends) != sendsBefore+1 || !strings.HasPrefix(m.sends[len(m.sends)-1], "reenroll:") {
|
||||
t.Fatalf("claimed re-enroll must send exactly one REENROLL email, got %v", m.sends)
|
||||
}
|
||||
})
|
||||
t.Run("unclaimed is a no-op (first-provision path)", func(t *testing.T) {
|
||||
|
||||
@@ -0,0 +1,167 @@
|
||||
package claim
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-hub/internal/notify"
|
||||
"golang.org/x/crypto/bcrypt"
|
||||
)
|
||||
|
||||
// ── R-295, HUB HALF — ONE NAME PER SECRET, AND THE MAIL NAMES THE PAGE THE MACHINE SHOWS ────────
|
||||
//
|
||||
// Two different secrets were both called „Visszaállító kód": the THREE-word code that gives a person
|
||||
// control of the dashboard, and the TEN-word code that opens the sealed off-site backups. They are
|
||||
// near-homographs of each other and of „Helyreállítási kód", and the collision cost a real code. The
|
||||
// box side shipped on 2026-08-10; the hub side was dropped twice and was the last place the retired
|
||||
// name survived — in the mails, which is the one surface a customer reads BEFORE they see any screen.
|
||||
//
|
||||
// The ruling these tests pin:
|
||||
// - the three-word dashboard code is „Beállító kód" everywhere;
|
||||
// - the ten-word escrow code is „Helyreállítási kód";
|
||||
// - „Visszaállító kód" is RETIRED;
|
||||
// - one secret in two situations keeps its NAME and changes its SENTENCE;
|
||||
// - the mail names the page the machine is ACTUALLY SHOWING.
|
||||
//
|
||||
// THIS IS NAMING, NOT FUNCTION — TestReenrollSplit_ChangesTheMailNotTheSecret is the pin that says
|
||||
// so. On the box side the companion pin is the controller's
|
||||
// TestResetCode_StillAcceptedOnTheSetupPage (claim_code_naming_test.go).
|
||||
|
||||
const retiredName = "Visszaállító kód"
|
||||
|
||||
// The three code-bearing mails all name the secret „Beállító kód", and none of them carries the
|
||||
// retired name. The escrow code's name must not appear in a dashboard-code mail either: naming both
|
||||
// secrets in one message is how a customer comes to believe they are the same thing.
|
||||
func TestFormatClaimEmail_OneNamePerSecret(t *testing.T) {
|
||||
for _, kind := range []EmailKind{EmailClaim, EmailReset, EmailReenroll} {
|
||||
subject, body := notify.FormatClaimEmail(string(kind), "c1", "example.hu", "alma-korte-szilva")
|
||||
whole := subject + "\n" + body
|
||||
|
||||
if !strings.Contains(body, "Beállító kód: alma-korte-szilva") {
|
||||
t.Errorf("%s: the secret must be labelled „Beállító kód”; body was:\n%s", kind, body)
|
||||
}
|
||||
if strings.Contains(whole, retiredName) {
|
||||
t.Errorf("%s: the retired name „%s” is back (subject or body)", kind, retiredName)
|
||||
}
|
||||
if strings.Contains(whole, "Helyreállítási kód") {
|
||||
t.Errorf("%s: names the ESCROW code in a dashboard-code mail — that is the collision", kind)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The two situations that deliver the SAME secret must not name the SAME page, because the machine
|
||||
// does not show the same page. A box that still has a password shows a login screen carrying an
|
||||
// „Elfelejtett jelszó" link; a REBUILT box has no password, renders „A szerver beállítása" and serves
|
||||
// no login page at all (controller web/claim.go:279 — `reset := s.authEnabled()`).
|
||||
//
|
||||
// Sending a re-enrolled customer to „Elfelejtett jelszó" names a route that is not on their screen.
|
||||
// That was the live defect, and this is the test that would have caught it.
|
||||
func TestFormatClaimEmail_NamesThePageTheMachineShows(t *testing.T) {
|
||||
_, resetBody := notify.FormatClaimEmail(string(EmailReset), "c1", "example.hu", "a-b-c")
|
||||
if !strings.Contains(resetBody, `"Elfelejtett jelszó"`) {
|
||||
t.Errorf("the forgot-password mail should name the page that IS on that customer's screen:\n%s", resetBody)
|
||||
}
|
||||
|
||||
_, reBody := notify.FormatClaimEmail(string(EmailReenroll), "c1", "example.hu", "a-b-c")
|
||||
if strings.Contains(reBody, "Elfelejtett jelszó") {
|
||||
t.Errorf("a REBUILT box serves no login page, so it has no „Elfelejtett jelszó” link:\n%s", reBody)
|
||||
}
|
||||
if !strings.Contains(reBody, `"A szerver beállítása"`) {
|
||||
t.Errorf("the re-enrol mail must name the page a rebuilt box actually shows:\n%s", reBody)
|
||||
}
|
||||
}
|
||||
|
||||
// The re-enrol mail must promise nothing about the customer's apps or backups. A clean-slate
|
||||
// reinstall is exactly the situation in which such a reassurance could be false, and this project has
|
||||
// spent four register rows (R-294, R-299, R-302, R-311) removing promises it could not see were still
|
||||
// true. Guarding the CLAIM rather than one phrasing of it: any sentence that says the backups are
|
||||
// unaffected would have to say so with one of these stems.
|
||||
func TestFormatClaimEmail_ReenrollPromisesNothingAboutTheData(t *testing.T) {
|
||||
_, body := notify.FormatClaimEmail(string(EmailReenroll), "c1", "example.hu", "a-b-c")
|
||||
for _, claimWord := range []string{"mentés", "biztonsági", "alkalmazás", "adataid", "visszaállíthat", "visszaszerezhet"} {
|
||||
if strings.Contains(body, claimWord) {
|
||||
t.Errorf("the re-enrol mail must not talk about data or backups (found %q):\n%s", claimWord, body)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// An unrecognised kind falls through to the first-setup mail. That default is SAFE (a rebuilt box
|
||||
// really is in a setup state) and it is deliberately asserted, because it is the failure mode of
|
||||
// adding a kind on the engine side and forgetting the template: nothing panics, nothing errors, and
|
||||
// the wrong-but-harmless mail goes out. Pinning it means a future reader knows it was chosen.
|
||||
func TestFormatClaimEmail_UnknownKindFallsBackToSetup(t *testing.T) {
|
||||
subject, body := notify.FormatClaimEmail("no-such-kind", "c1", "example.hu", "a-b-c")
|
||||
if !strings.Contains(subject, "Elindult a Felhom szervered") {
|
||||
t.Errorf("unknown kind should fall back to the first-setup mail, got subject %q", subject)
|
||||
}
|
||||
if strings.Contains(subject+body, retiredName) {
|
||||
t.Errorf("the fallback carries the retired name")
|
||||
}
|
||||
}
|
||||
|
||||
// THE ACCEPTANCE PIN. Splitting the mail KIND must not touch the secret: ReissueForReenroll still
|
||||
// rotates the generation, still stores a bcrypt hash that verifies the emailed code, and still stores
|
||||
// no plaintext — byte-for-byte the same custody as the reset path it was split out of. A rename that
|
||||
// quietly broke acceptance would be far worse than the collision it fixes.
|
||||
func TestReenrollSplit_ChangesTheMailNotTheSecret(t *testing.T) {
|
||||
e, st, m := newTestEngine(t)
|
||||
cc := cust()
|
||||
if _, err := e.EnsureIssued(cc); err != nil {
|
||||
t.Fatalf("EnsureIssued: %v", err)
|
||||
}
|
||||
if err := e.MarkClaimed(cc); err != nil {
|
||||
t.Fatalf("MarkClaimed: %v", err)
|
||||
}
|
||||
before, err := st.GetClaim(cc.CustomerID)
|
||||
if err != nil {
|
||||
t.Fatalf("GetClaim: %v", err)
|
||||
}
|
||||
|
||||
gen, reissued, err := e.ReissueForReenroll(cc)
|
||||
if err != nil || !reissued {
|
||||
t.Fatalf("ReissueForReenroll: gen=%d reissued=%v err=%v", gen, reissued, err)
|
||||
}
|
||||
|
||||
// It is the re-enrol mail, not the reset mail — the whole point of the split.
|
||||
last := m.sends[len(m.sends)-1]
|
||||
if !strings.HasPrefix(last, string(EmailReenroll)+":") {
|
||||
t.Errorf("a re-enrolled box must get the reenroll mail, got %q", last)
|
||||
}
|
||||
|
||||
// …and the secret behaves exactly as before: generation advanced, the stored hash verifies the
|
||||
// newly emailed code, the previous code no longer verifies, and no plaintext is persisted.
|
||||
after, err := st.GetClaim(cc.CustomerID)
|
||||
if err != nil {
|
||||
t.Fatalf("GetClaim after: %v", err)
|
||||
}
|
||||
if after.Generation != before.Generation+1 {
|
||||
t.Errorf("generation %d → %d, want +1", before.Generation, after.Generation)
|
||||
}
|
||||
if bcrypt.CompareHashAndPassword([]byte(after.CodeHash), []byte(m.lastCode)) != nil {
|
||||
t.Error("the stored hash does not verify the emailed code — acceptance moved")
|
||||
}
|
||||
if strings.Contains(after.CodeHash, m.lastCode) {
|
||||
t.Error("the stored hash contains the plaintext code")
|
||||
}
|
||||
}
|
||||
|
||||
// RequestReset — a customer who still has a password — must keep sending the RESET mail. The split
|
||||
// must not have swept the genuine forgot-password path along with it: that customer IS looking at a
|
||||
// login page, and „Elfelejtett jelszó" is the right thing to name for them.
|
||||
func TestRequestReset_StillSendsTheResetMail(t *testing.T) {
|
||||
e, _, m := newTestEngine(t)
|
||||
cc := cust()
|
||||
if _, err := e.EnsureIssued(cc); err != nil {
|
||||
t.Fatalf("EnsureIssued: %v", err)
|
||||
}
|
||||
if err := e.MarkClaimed(cc); err != nil {
|
||||
t.Fatalf("MarkClaimed: %v", err)
|
||||
}
|
||||
if err := e.RequestReset(cc); err != nil {
|
||||
t.Fatalf("RequestReset: %v", err)
|
||||
}
|
||||
last := m.sends[len(m.sends)-1]
|
||||
if !strings.HasPrefix(last, string(EmailReset)+":") {
|
||||
t.Errorf("a forgot-password request must still send the reset mail, got %q", last)
|
||||
}
|
||||
}
|
||||
@@ -69,7 +69,7 @@ Message: %s`, customerID, eventType, severity, now, message)
|
||||
// customerMessages maps event_type → Hungarian customer message.
|
||||
var customerMessages = map[string]string{
|
||||
// Customer-claim arc (v0.50.0)
|
||||
"claim_lockout": "Túl sok hibás beállító/visszaállító kód próbálkozás történt — a beállító oldal 15 percre zárolva lett. Ha nem te próbálkoztál, jelezd az üzemeltetőnek.",
|
||||
"claim_lockout": "Túl sok hibás beállító kód próbálkozás történt — a beállító oldal 15 percre zárolva lett. Ha nem te próbálkoztál, jelezd az üzemeltetőnek.",
|
||||
// Backup events
|
||||
"backup_completed": "A biztonsági mentés sikeresen elkészült.",
|
||||
"backup_failed": "A biztonsági mentés sikertelen! Kérjük, ellenőrizd a rendszert.",
|
||||
@@ -207,18 +207,38 @@ Felhom.eu monitoring`
|
||||
// ──────────────────────────────────────────────────────────────────────
|
||||
|
||||
// FormatClaimEmail returns (subject, textBody) for a claim-arc email. kind is one of
|
||||
// "claim" | "reset" | "claimed" (claim.EmailKind values). The code appears ONLY in the
|
||||
// "claim" | "reset" | "reenroll" | "claimed" (claim.EmailKind values). The code appears ONLY in the
|
||||
// returned body — callers must never log it.
|
||||
//
|
||||
// R-295, HUB HALF (2026-08-13). ONE NAME PER SECRET, and it is „Beállító kód".
|
||||
//
|
||||
// The three-word code that gives a person control of the DASHBOARD is „Beállító kód" everywhere —
|
||||
// in this mail, on the operator button, and on the box's own page, which has said so since the
|
||||
// controller half shipped on 2026-08-10. The TEN-word code that opens the sealed backups is
|
||||
// „Helyreállítási kód" and is a different secret entirely. **„Visszaállító kód" is retired**: it was
|
||||
// a near-homograph of „Helyreállítási kód", the collision cost a real code, and the hub was the last
|
||||
// place it survived — this half was dropped twice before it was finished.
|
||||
//
|
||||
// The rule the three branches below follow: ONE SECRET IN TWO SITUATIONS KEEPS ITS NAME, AND THE
|
||||
// SENTENCE AROUND IT CHANGES. „reset" and „reenroll" carry the identical secret under the identical
|
||||
// name; they differ only in which page the customer will actually be looking at.
|
||||
//
|
||||
// THIS IS NAMING, NOT FUNCTION. No acceptance logic moved: the code is minted, hashed, rotated,
|
||||
// capped and consumed exactly as before, and a reset code is still accepted on the setup page.
|
||||
// Pinned by TestFormatClaimEmail_OneNamePerSecret and, on the box side, by the controller's
|
||||
// claim_code_naming_test.go.
|
||||
func FormatClaimEmail(kind, customerID, domain, code string) (string, string) {
|
||||
dashboardURL := "https://felhom." + domain
|
||||
switch kind {
|
||||
case "reset":
|
||||
subject := "[Felhom] Jelszó-visszaállítási kód"
|
||||
// The box HAS a password: the customer asked for a reset from the login screen, so the
|
||||
// „Elfelejtett jelszó" link IS on their screen and naming it is correct here.
|
||||
subject := "[Felhom] Beállító kód a jelszavad visszaállításához"
|
||||
body := fmt.Sprintf(`Kedves Ügyfél!
|
||||
|
||||
Jelszó-visszaállítást kértél a Felhom vezérlőpultodhoz.
|
||||
|
||||
Visszaállító kód: %s
|
||||
Beállító kód: %s
|
||||
|
||||
A kód 72 óráig érvényes, és egyszer használható fel. Add meg a vezérlőpult
|
||||
"Elfelejtett jelszó" oldalán, majd válassz új jelszót:
|
||||
@@ -227,6 +247,32 @@ A kód 72 óráig érvényes, és egyszer használható fel. Add meg a vezérlő
|
||||
|
||||
Ha nem te kérted, hagyd figyelmen kívül — a jelenlegi jelszavad változatlan.
|
||||
|
||||
Üdvözlettel,
|
||||
Felhom.eu`, code, dashboardURL)
|
||||
return subject, body
|
||||
case "reenroll":
|
||||
// The box was rebuilt and has NO password, so it shows „A szerver beállítása" and serves no
|
||||
// login page — there is no „Elfelejtett jelszó" link to send anyone to. Same secret, same
|
||||
// name, the page the machine is actually showing.
|
||||
//
|
||||
// It deliberately says NOTHING about the apps or the backups. A clean-slate reinstall is
|
||||
// exactly the situation in which such a reassurance could be false, and this project has
|
||||
// spent four register rows removing promises it could not see were still true.
|
||||
subject := "[Felhom] Új beállító kód — újratelepült a szervered"
|
||||
body := fmt.Sprintf(`Kedves Ügyfél!
|
||||
|
||||
A Felhom szervered újratelepült, ezért a vezérlőpultod belépését újra be kell
|
||||
állítani. A korábbi jelszavad már nem érvényes.
|
||||
|
||||
Beállító kód: %s
|
||||
|
||||
A kód 72 óráig érvényes, és egyszer használható fel. Nyisd meg a vezérlőpultot —
|
||||
"A szerver beállítása" oldal fogad —, add meg a kódot, majd válassz új jelszót:
|
||||
|
||||
%s
|
||||
|
||||
Ha nem te telepítetted újra a szervered, vedd fel a kapcsolatot az üzemeltetővel.
|
||||
|
||||
Üdvözlettel,
|
||||
Felhom.eu`, code, dashboardURL)
|
||||
return subject, body
|
||||
|
||||
@@ -75,8 +75,13 @@ func TestCustomerPage_ClaimCardStates(t *testing.T) {
|
||||
if !strings.Contains(html, ">Claimed ") {
|
||||
t.Error("missing the Claimed chip")
|
||||
}
|
||||
if !strings.Contains(html, "Visszaállító kód küldése") {
|
||||
t.Error("claimed state should offer the reset-code button")
|
||||
// R-295: the button says „Beállító kód küldése" — ONE name per secret. It used to read
|
||||
// „Visszaállító kód küldése", a near-homograph of the escrow „Helyreállítási kód".
|
||||
if !strings.Contains(html, "Beállító kód küldése") {
|
||||
t.Error("claimed state should offer the reset-code button, named „Beállító kód küldése”")
|
||||
}
|
||||
if strings.Contains(html, "Visszaállító kód") {
|
||||
t.Error("the retired name „Visszaállító kód” is back on the customer page")
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -776,8 +776,9 @@ func (s *Server) handleConfigUpdate(w http.ResponseWriter, r *http.Request, cust
|
||||
}
|
||||
|
||||
// handleClaimResend (v0.50.0, customer-claim arc) rotates the claim/reset code and re-sends it to
|
||||
// the REGISTERED customer address — the operator "Kód újraküldése" / "Visszaállító kód küldése"
|
||||
// button. The old code stops verifying immediately (single active code); the fresh hash reaches
|
||||
// the REGISTERED customer address — the operator "Kód újraküldése" / "Beállító kód küldése"
|
||||
// button (R-295: the button used to read „Visszaállító kód küldése"; that name is retired).
|
||||
// The old code stops verifying immediately (single active code); the fresh hash reaches
|
||||
// the box on its next report ACK (no config bump needed). No plaintext is ever rendered or logged.
|
||||
func (s *Server) handleClaimResend(w http.ResponseWriter, r *http.Request, customerID string) {
|
||||
if s.claimEngine == nil {
|
||||
|
||||
@@ -263,6 +263,175 @@ func (s *Server) hostNetwork(host *store.Host, reportJSON string) hostNetworkVie
|
||||
return v
|
||||
}
|
||||
|
||||
// ── R-319 — the guest-network watchdog gets a reader ────────────────────────────────────────────
|
||||
//
|
||||
// FIRST OF THE R-264 READERS. The agent has reported `guest_net` on every heartbeat since v0.92.0
|
||||
// (R-54, 2026-07-21) and the hub — the component that emails the operator — modelled NONE of it: the
|
||||
// string `guest_net` occurred nowhere in this repository. It was stored as raw text inside
|
||||
// `report_json` and read by nothing.
|
||||
//
|
||||
// WHY THIS ONE FIRST. It has a live incident behind it: a killed `dhclient` in a guest took a tunnel
|
||||
// down for 1 h 15 m with nobody told (`audits/INCIDENT-guest-dhclient-killed-2026-07-20.md`), and the
|
||||
// watchdog built afterwards has been reporting exactly that condition ever since — to a hub that
|
||||
// modelled none of it. R-264 named it "the strongest candidate of the twenty-one".
|
||||
//
|
||||
// WHAT IS ACTUALLY BEING READ, and it is NOT just the current state. The signal is
|
||||
// `heals_last_hour`: a guest whose network the watchdog keeps REPAIRING is healthy at every instant
|
||||
// anyone looks and is nevertheless failing. Rendering only `state` would give that machine a green
|
||||
// tick — which is the exact shape of the defect that drew a failed disk as a healthy empty disk.
|
||||
// RepairCount is therefore surfaced beside the state, not behind it.
|
||||
//
|
||||
// AN UNKNOWN IS NEVER DRAWN AS HEALTHY. Three distinct absences are kept apart, following the
|
||||
// companion-flag convention this project settled in August (`hostNetworkView` above is the sibling):
|
||||
// - AgentTooOld — the agent predates v0.92.0, so absence is EXPECTED but still tells us nothing;
|
||||
// - Reported=false — a new-enough agent sent no stanza (watchdog disabled, or a report that
|
||||
// predates the feature on this box);
|
||||
// - a guest whose `state` is empty or unrecognised → rendered "unknown", never healthy.
|
||||
// A malformed stanza degrades to Reported=false and must never 500 the page.
|
||||
|
||||
// minAgentForGuestNet is the agent release that first reported `guest_net` (v0.92.0, R-54). Below it
|
||||
// the stanza is absent BY CONSTRUCTION, so its absence is unknown-and-expected rather than a finding.
|
||||
const minAgentForGuestNet = "0.92.0"
|
||||
|
||||
// guestNetGuestView is one owned guest's network health as the watchdog last saw it.
|
||||
type guestNetGuestView struct {
|
||||
VMID int
|
||||
State string // healthy | unhealthy | static_fault | unknown ("" → unknown)
|
||||
Mode string // dhcp | static | unknown
|
||||
IP string
|
||||
HasRoute bool
|
||||
DHClientAlive bool
|
||||
// RepairCount is `heals_last_hour` — THE SIGNAL. A machine repairing itself over and over is
|
||||
// telling you something that its current state cannot.
|
||||
RepairCount int
|
||||
LastHealAt string
|
||||
Damped bool
|
||||
Message string
|
||||
// Healed / HealSucceeded are carried DELIBERATELY, and together. R-260 is this project's
|
||||
// warning: the OOB decoder mirrored five of the agent's eight fields, and the three it dropped
|
||||
// included the one that decided the question the checker existed to answer. A repair that FAILED
|
||||
// is a different fact from a repair that worked, and the count alone cannot express it — six
|
||||
// successful repairs is a nuisance, six FAILED ones is a guest that is down right now.
|
||||
Healed bool
|
||||
HealSucceeded bool
|
||||
}
|
||||
|
||||
// HealFailed reports that the watchdog TRIED to repair this guest and did not succeed. Kept separate
|
||||
// from State because the two can disagree: the sweep that failed to heal is not necessarily the sweep
|
||||
// that set the state.
|
||||
func (g guestNetGuestView) HealFailed() bool { return g.Healed && !g.HealSucceeded }
|
||||
|
||||
// Unknown reports whether this guest's state is one the watchdog did not positively assert. An empty
|
||||
// or unrecognised state is drawn as unknown; it is never allowed to fall through to the healthy
|
||||
// branch. Value receiver — a pointer receiver compiles, vets, passes the suite and 500s at render.
|
||||
func (g guestNetGuestView) Unknown() bool {
|
||||
switch g.State {
|
||||
case "healthy", "unhealthy", "static_fault":
|
||||
return false
|
||||
default:
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
// Repairing reports whether the watchdog has had to repair this guest inside the last hour. This is
|
||||
// deliberately independent of State: the whole point is that a REPAIRED guest reads healthy.
|
||||
func (g guestNetGuestView) Repairing() bool { return g.RepairCount > 0 }
|
||||
|
||||
// guestNetView is the Guest network card's whole view-model.
|
||||
type guestNetView struct {
|
||||
// Reported is the ONLY thing that licenses drawing any health at all. False → unknown.
|
||||
Reported bool
|
||||
// AgentTooOld distinguishes "this agent cannot report it" from "a capable agent said nothing".
|
||||
// Both render as unknown; they need different words, and conflating them is how an operator
|
||||
// starts ignoring the card.
|
||||
AgentTooOld bool
|
||||
CheckedAt string
|
||||
Guests []guestNetGuestView
|
||||
// RepairingCount / UnhealthyCount drive the card's summary badge. Counted rather than derived in
|
||||
// the template: template logic that computes a verdict is logic nobody tests.
|
||||
RepairingCount int
|
||||
UnhealthyCount int
|
||||
UnknownCount int
|
||||
// HealFailedCount — repairs ATTEMPTED and not succeeded. Counted separately from
|
||||
// RepairingCount: a failing repair is a harder fact than a frequent one.
|
||||
HealFailedCount int
|
||||
}
|
||||
|
||||
// Degraded is true when any owned guest is unhealthy OR is being repeatedly repaired. The repair leg
|
||||
// is the one that matters: without it a guest the watchdog fixes every ten minutes reports "healthy"
|
||||
// for ever.
|
||||
func (v guestNetView) Degraded() bool {
|
||||
return v.UnhealthyCount > 0 || v.RepairingCount > 0 || v.HealFailedCount > 0
|
||||
}
|
||||
|
||||
// parseGuestNet extracts the `guest_net` stanza. A missing or malformed body yields Reported=false
|
||||
// (unknown), never an error and never a partial claim of health.
|
||||
func parseGuestNet(reportJSON string) guestNetView {
|
||||
v := guestNetView{Guests: []guestNetGuestView{}}
|
||||
if reportJSON == "" {
|
||||
return v
|
||||
}
|
||||
var body struct {
|
||||
GuestNet *struct {
|
||||
CheckedAt string `json:"checked_at"`
|
||||
Guests []struct {
|
||||
VMID int `json:"vmid"`
|
||||
State string `json:"state"`
|
||||
Mode string `json:"mode"`
|
||||
IP string `json:"ip"`
|
||||
HasRoute bool `json:"has_route"`
|
||||
DHClientAlive bool `json:"dhclient_alive"`
|
||||
HealsLastHour int `json:"heals_last_hour"`
|
||||
Healed bool `json:"healed"`
|
||||
HealSucceeded bool `json:"heal_succeeded"`
|
||||
LastHealAt string `json:"last_heal_at"`
|
||||
Damped bool `json:"damped"`
|
||||
Message string `json:"message"`
|
||||
} `json:"guests"`
|
||||
} `json:"guest_net"`
|
||||
}
|
||||
// A decode error is NOT propagated: one box's bad field must not break the page for the rest of
|
||||
// the fleet. It degrades to Reported=false, which renders as unknown — the safe direction.
|
||||
if err := json.Unmarshal([]byte(reportJSON), &body); err != nil || body.GuestNet == nil {
|
||||
return v
|
||||
}
|
||||
v.Reported = true
|
||||
v.CheckedAt = body.GuestNet.CheckedAt
|
||||
for _, g := range body.GuestNet.Guests {
|
||||
gv := guestNetGuestView{
|
||||
VMID: g.VMID, State: g.State, Mode: g.Mode, IP: g.IP,
|
||||
HasRoute: g.HasRoute, DHClientAlive: g.DHClientAlive,
|
||||
RepairCount: g.HealsLastHour, LastHealAt: g.LastHealAt,
|
||||
Damped: g.Damped, Message: g.Message,
|
||||
Healed: g.Healed, HealSucceeded: g.HealSucceeded,
|
||||
}
|
||||
switch {
|
||||
case gv.Unknown():
|
||||
v.UnknownCount++
|
||||
case gv.State == "unhealthy" || gv.State == "static_fault":
|
||||
v.UnhealthyCount++
|
||||
}
|
||||
if gv.Repairing() {
|
||||
v.RepairingCount++
|
||||
}
|
||||
if gv.HealFailed() {
|
||||
v.HealFailedCount++
|
||||
}
|
||||
v.Guests = append(v.Guests, gv)
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
// guestNet builds the card's view-model, applying the version gate before the report is consulted so
|
||||
// that an old agent's silence is named as such rather than rendered as a finding.
|
||||
func (s *Server) guestNet(host *store.Host, reportJSON string) guestNetView {
|
||||
if host.AgentVersion != "" && semver.Valid(host.AgentVersion) &&
|
||||
semver.Compare(host.AgentVersion, minAgentForGuestNet) < 0 {
|
||||
return guestNetView{Guests: []guestNetGuestView{}, AgentTooOld: true}
|
||||
}
|
||||
return parseGuestNet(reportJSON)
|
||||
}
|
||||
|
||||
// storageTargetView is the rich per-drive row the host-detail Storage Targets table renders:
|
||||
// fill %, role/state, thin-pool, and SMART health/temp/wear. Parsed from the latest report's
|
||||
// storage_targets[] (the full hostStorageTarget wire shape lives in the api package; this view
|
||||
@@ -533,6 +702,9 @@ func (s *Server) hostDetailData(host *store.Host, r *http.Request) map[string]in
|
||||
// v0.85.0 Network — the host's addresses + its WireGuard allocation.
|
||||
network := s.hostNetwork(host, reportJSON)
|
||||
|
||||
// R-319 Guest network — the R-54 watchdog's per-guest verdict AND its repair count.
|
||||
guestNet := s.guestNet(host, reportJSON)
|
||||
|
||||
// v0.84.0 Console access — presence + username + set_at ONLY. GetHostRecoveryMeta cannot carry
|
||||
// the secret (its query does not select the column); the plaintext reaches the operator solely
|
||||
// through POST /hosts/{id}/reveal-recovery-credential.
|
||||
@@ -573,6 +745,7 @@ func (s *Server) hostDetailData(host *store.Host, r *http.Request) map[string]in
|
||||
// v0.84.0 break-glass Console access card. NEVER add a key holding the secret.
|
||||
// v0.85.0 Network card (addresses + WireGuard allocation/confirmation).
|
||||
"Network": network,
|
||||
"GuestNet": guestNet,
|
||||
"RecoveryVaulted": recoveryMeta != nil,
|
||||
"RecoveryUsername": func() string {
|
||||
if recoveryMeta != nil {
|
||||
|
||||
@@ -0,0 +1,259 @@
|
||||
package web
|
||||
|
||||
// Guest-network card (R-319) — the first R-264 reader.
|
||||
//
|
||||
// THE FIXTURES BELOW ARE THE REAL WIRE. `liveGuestNetJSON` is the `guest_net` stanza copied verbatim
|
||||
// out of `demo-felhom-8363b5`'s newest row in the live hub's `host_reports` table on 2026-08-13
|
||||
// (agent 0.129.0). Testing against a hand-written shape would have proved only that the parser
|
||||
// matches my own idea of the format — and the defect this whole class comes from (R-260) was exactly
|
||||
// a hub-side struct that did not match what the agent actually sends.
|
||||
//
|
||||
// Every test drives ServeHTTP, so a template gate that never renders is visible here. That is the
|
||||
// seam-wiring rule: handler tests prove nothing about reachability, and this project has shipped four
|
||||
// features whose entry point was never wired.
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The real stanza: one owned guest, healthy, DHCP, no repairs. `heals_last_hour` and friends are
|
||||
// ABSENT because the agent marks them `omitempty` and this box has never needed a repair — which is
|
||||
// itself the shape scenario A has to survive.
|
||||
const liveGuestNetJSON = `{
|
||||
"host": {"cpu_percent": 4.0, "memory_percent": 30.0, "disk_percent": 20.0},
|
||||
"guest_net": {
|
||||
"checked_at": "2026-08-13T08:15:33Z",
|
||||
"guests": [
|
||||
{"vmid": 9201, "state": "healthy", "mode": "dhcp", "ip": "192.168.0.149",
|
||||
"has_route": true, "dhclient_alive": true, "checked_at": "2026-08-13T08:14:30Z",
|
||||
"message": "address, default route and dhclient all present"}
|
||||
]
|
||||
}
|
||||
}`
|
||||
|
||||
// The same box after the watchdog has had to keep fixing it — the R-54 heal fields populated. This is
|
||||
// the shape the incident of 2026-07-20 would have produced had the watchdog existed then.
|
||||
const repairingGuestNetJSON = `{
|
||||
"host": {"cpu_percent": 4.0, "memory_percent": 30.0, "disk_percent": 20.0},
|
||||
"guest_net": {
|
||||
"checked_at": "2026-08-13T08:15:33Z",
|
||||
"guests": [
|
||||
{"vmid": 9201, "state": "healthy", "mode": "dhcp", "ip": "192.168.0.149",
|
||||
"has_route": true, "dhclient_alive": true, "checked_at": "2026-08-13T08:14:30Z",
|
||||
"healed": true, "heal_succeeded": true, "last_heal_at": "2026-08-13T08:09:12Z",
|
||||
"heals_last_hour": 6, "message": "dhclient was absent; restarted"}
|
||||
]
|
||||
}
|
||||
}`
|
||||
|
||||
// A report from a capable agent with NO guest_net stanza at all — the watchdog switched off, or a
|
||||
// report predating the feature on this box.
|
||||
const silentGuestNetJSON = `{"host": {"cpu_percent": 4.0, "memory_percent": 30.0, "disk_percent": 20.0}}`
|
||||
|
||||
// The stanza arrives, but its contents are not what the hub expects: `guests` is an object where an
|
||||
// array belongs, and `heals_last_hour` is a string. This is what a wire drift or a truncated write
|
||||
// looks like from the hub's side.
|
||||
const malformedGuestNetJSON = `{
|
||||
"host": {"cpu_percent": 4.0},
|
||||
"guest_net": {"checked_at": "2026-08-13T08:15:33Z", "guests": {"vmid": "nine-two-oh-one"}}
|
||||
}`
|
||||
|
||||
// ── A — a machine reporting healthy guest networking, no repairs → shown as healthy ─────────────
|
||||
//
|
||||
// WRONG OUTCOME GUARDED: an empty or alarming state on a machine that is fine. A card that cried
|
||||
// unknown on every healthy box would be switched off within a week, and then the B case below would
|
||||
// never be seen either.
|
||||
func TestGuestNet_A_HealthyRendersHealthy(t *testing.T) {
|
||||
s, st, _ := newRevealServer(t)
|
||||
cookie, _ := newRevealSession(t, s)
|
||||
seedNetHost(t, st, "demo-felhom-8363b5", "0.129.0", liveGuestNetJSON, "")
|
||||
|
||||
body := getHostPage(t, s, cookie, "demo-felhom-8363b5")
|
||||
|
||||
if !strings.Contains(body, "Guest network") {
|
||||
t.Fatal("no Guest network card — the reader shows nothing")
|
||||
}
|
||||
if !strings.Contains(body, "192.168.0.149") {
|
||||
t.Error("the guest's address did not reach the page")
|
||||
}
|
||||
if !strings.Contains(body, "badge-ok") {
|
||||
t.Error("a healthy guest must render as healthy, not as an empty or alarming state")
|
||||
}
|
||||
if strings.Contains(body, "needs attention") {
|
||||
t.Error("a machine that is fine is being alarmed on")
|
||||
}
|
||||
// The unknown branches must NOT fire for a box that reported properly.
|
||||
if strings.Contains(body, "does not run the guest-network watchdog") ||
|
||||
strings.Contains(body, "reported no guest-network state") {
|
||||
t.Error("a reporting box rendered one of the unknown sentences")
|
||||
}
|
||||
}
|
||||
|
||||
// ── B — a machine whose watchdog has repaired the guest repeatedly ──────────────────────────────
|
||||
//
|
||||
// WRONG OUTCOME GUARDED: a green tick because the CURRENT state is fine. This is the exact shape of
|
||||
// the failed-disk-drawn-as-a-healthy-empty-disk defect, and it is the reason this card exists at all:
|
||||
// `state` says "healthy" in this fixture, because between repairs it IS.
|
||||
func TestGuestNet_B_RepeatedRepairsAreVisible(t *testing.T) {
|
||||
s, st, _ := newRevealServer(t)
|
||||
cookie, _ := newRevealSession(t, s)
|
||||
seedNetHost(t, st, "demo-felhom-8363b5", "0.129.0", repairingGuestNetJSON, "")
|
||||
|
||||
body := getHostPage(t, s, cookie, "demo-felhom-8363b5")
|
||||
|
||||
if !strings.Contains(body, "needs attention") {
|
||||
t.Fatal("a guest repaired 6 times in an hour is reported as fine — the whole point of the card")
|
||||
}
|
||||
if !strings.Contains(body, "had their networking repaired in the last hour") {
|
||||
t.Error("the climbing-repairs sentence is missing")
|
||||
}
|
||||
if !strings.Contains(body, ">6<") {
|
||||
t.Error("the repair COUNT is not rendered; 'some repairs' is not the signal, the number is")
|
||||
}
|
||||
if !strings.Contains(body, "2026-08-13T08:09:12Z") {
|
||||
t.Error("the last repair time is not offered (it is in the title attribute)")
|
||||
}
|
||||
}
|
||||
|
||||
// ── C — a machine that does not report the fact at all → drawn as unknown, never healthy ────────
|
||||
//
|
||||
// WRONG OUTCOME GUARDED: absence read as good news. THE ONE THAT MATTERS. Two distinct absences are
|
||||
// checked because they need different words: an agent that cannot report it, and a capable agent that
|
||||
// said nothing.
|
||||
func TestGuestNet_C_SilentIsUnknownNotHealthy(t *testing.T) {
|
||||
t.Run("capable agent, no stanza", func(t *testing.T) {
|
||||
s, st, _ := newRevealServer(t)
|
||||
cookie, _ := newRevealSession(t, s)
|
||||
seedNetHost(t, st, "demo-hp-bb76ea", "0.129.0", silentGuestNetJSON, "")
|
||||
|
||||
body := getHostPage(t, s, cookie, "demo-hp-bb76ea")
|
||||
|
||||
if !strings.Contains(body, "reported no guest-network state") {
|
||||
t.Fatal("a silent box must SAY it is unknown")
|
||||
}
|
||||
if strings.Contains(body, "badge-ok") && strings.Contains(body, "Guest network") {
|
||||
assertNoHealthyBadgeInGuestNetCard(t, body)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("agent too old to report it", func(t *testing.T) {
|
||||
s, st, _ := newRevealServer(t)
|
||||
cookie, _ := newRevealSession(t, s)
|
||||
// 0.91.0 predates the R-54 watchdog (v0.92.0): the absence is expected AND still unknown.
|
||||
seedNetHost(t, st, "old-agent-box", "0.91.0", silentGuestNetJSON, "")
|
||||
|
||||
body := getHostPage(t, s, cookie, "old-agent-box")
|
||||
|
||||
if !strings.Contains(body, "does not run the guest-network watchdog") {
|
||||
t.Fatal("an old agent's silence must be named as an old agent's silence")
|
||||
}
|
||||
if !strings.Contains(body, "<strong>unknown</strong>, not healthy") {
|
||||
t.Error("the unknown-is-not-healthy sentence is missing")
|
||||
}
|
||||
assertNoHealthyBadgeInGuestNetCard(t, body)
|
||||
})
|
||||
}
|
||||
|
||||
// ── D — the fact arrives malformed → unknown, and the hub does not 500 ──────────────────────────
|
||||
//
|
||||
// WRONG OUTCOME GUARDED: a page that breaks on one machine's bad field. getHostPage fails the test on
|
||||
// any non-200, so surviving the call IS half the assertion.
|
||||
func TestGuestNet_D_MalformedIsUnknownAndDoesNotBreakThePage(t *testing.T) {
|
||||
s, st, _ := newRevealServer(t)
|
||||
cookie, _ := newRevealSession(t, s)
|
||||
seedNetHost(t, st, "demo-felhom-8363b5", "0.129.0", malformedGuestNetJSON, "")
|
||||
|
||||
body := getHostPage(t, s, cookie, "demo-felhom-8363b5") // 200 or the test dies here
|
||||
|
||||
if !strings.Contains(body, "reported no guest-network state") {
|
||||
t.Fatal("a malformed stanza must degrade to unknown")
|
||||
}
|
||||
assertNoHealthyBadgeInGuestNetCard(t, body)
|
||||
}
|
||||
|
||||
// A guest whose state string is one the watchdog never asserts (a future value, or a truncated field)
|
||||
// is drawn as unknown rather than falling through to the healthy branch. The switch in
|
||||
// guestNetGuestView.Unknown is an ALLOW-LIST for exactly this reason: an unrecognised value defaults
|
||||
// to unknown, and adding a state to the agent cannot silently paint it green here.
|
||||
func TestGuestNet_UnrecognisedStateIsUnknown(t *testing.T) {
|
||||
s, st, _ := newRevealServer(t)
|
||||
cookie, _ := newRevealSession(t, s)
|
||||
seedNetHost(t, st, "demo-felhom-8363b5", "0.129.0", `{
|
||||
"host": {"cpu_percent": 1.0},
|
||||
"guest_net": {"checked_at": "2026-08-13T08:15:33Z",
|
||||
"guests": [{"vmid": 9201, "state": "quantum", "mode": "dhcp", "ip": "10.0.0.5"}]}}`, "")
|
||||
|
||||
body := getHostPage(t, s, cookie, "demo-felhom-8363b5")
|
||||
|
||||
if !strings.Contains(body, "partly unknown") {
|
||||
t.Fatal("an unrecognised guest state must summarise as unknown, not healthy")
|
||||
}
|
||||
}
|
||||
|
||||
// The empty-list case is deliberately NOT the same as silence: the agent's own contract says a stanza
|
||||
// with a fresh checked_at and no guests means "the watchdog ran and found nothing to report", and that
|
||||
// must stay distinguishable from "the watchdog is not wired" — the shape the v0.91.0 inert seam hid
|
||||
// behind.
|
||||
func TestGuestNet_EmptyListIsNotSilence(t *testing.T) {
|
||||
s, st, _ := newRevealServer(t)
|
||||
cookie, _ := newRevealSession(t, s)
|
||||
seedNetHost(t, st, "demo-felhom-8363b5", "0.129.0",
|
||||
`{"host": {"cpu_percent": 1.0}, "guest_net": {"checked_at": "2026-08-13T08:15:33Z"}}`, "")
|
||||
|
||||
body := getHostPage(t, s, cookie, "demo-felhom-8363b5")
|
||||
|
||||
if !strings.Contains(body, "The watchdog ran and has no guest to report") {
|
||||
t.Fatal("an empty guest list must say the watchdog RAN")
|
||||
}
|
||||
if strings.Contains(body, "reported no guest-network state") {
|
||||
t.Error("an empty list was collapsed into silence — the two are different facts")
|
||||
}
|
||||
}
|
||||
|
||||
// assertNoHealthyBadgeInGuestNetCard checks that the Guest network card contains no healthy badge.
|
||||
// It slices the card out of the page rather than searching the whole body, because `badge-ok` occurs
|
||||
// all over a host page (capabilities, storage, WireGuard) and a whole-page search would pass
|
||||
// vacuously — which is the "instrument that can drop results silently" trap.
|
||||
func assertNoHealthyBadgeInGuestNetCard(t *testing.T, body string) {
|
||||
t.Helper()
|
||||
i := strings.Index(body, "Guest network")
|
||||
if i < 0 {
|
||||
t.Fatal("no Guest network card to slice — the assertion would have passed vacuously")
|
||||
}
|
||||
card := body[i:]
|
||||
if j := strings.Index(card, "</section>"); j >= 0 {
|
||||
card = card[:j]
|
||||
} else {
|
||||
t.Fatal("could not find the end of the Guest network card")
|
||||
}
|
||||
if strings.Contains(card, "badge-ok") {
|
||||
t.Errorf("an UNKNOWN guest-network state rendered a healthy badge:\n%s", card)
|
||||
}
|
||||
}
|
||||
|
||||
// A repair that was ATTEMPTED and FAILED is a harder fact than a repair that worked, and the count
|
||||
// alone cannot express it: six successful repairs is a nuisance, six failed ones is a guest that is
|
||||
// down right now. Dropping `heal_succeeded` from the decoder would be R-260 exactly — the OOB decoder
|
||||
// mirrored five of eight fields and the three it dropped included the deciding one.
|
||||
func TestGuestNet_FailedRepairIsDistinctFromFrequentRepair(t *testing.T) {
|
||||
s, st, _ := newRevealServer(t)
|
||||
cookie, _ := newRevealSession(t, s)
|
||||
seedNetHost(t, st, "demo-felhom-8363b5", "0.129.0", `{
|
||||
"host": {"cpu_percent": 1.0},
|
||||
"guest_net": {"checked_at": "2026-08-13T08:15:33Z",
|
||||
"guests": [{"vmid": 9201, "state": "unhealthy", "mode": "dhcp",
|
||||
"has_route": false, "dhclient_alive": false,
|
||||
"healed": true, "heal_succeeded": false, "heals_last_hour": 3,
|
||||
"message": "dhclient restart did not restore the lease"}]}}`, "")
|
||||
|
||||
body := getHostPage(t, s, cookie, "demo-felhom-8363b5")
|
||||
|
||||
if !strings.Contains(body, "repair failed") {
|
||||
t.Fatal("a repair that was attempted and failed is not distinguished from one that worked")
|
||||
}
|
||||
if !strings.Contains(body, "needs attention") {
|
||||
t.Error("a guest whose repair failed must summarise as needing attention")
|
||||
}
|
||||
assertNoHealthyBadgeInGuestNetCard(t, body)
|
||||
}
|
||||
@@ -473,7 +473,7 @@
|
||||
{{if .Claim}}
|
||||
<form method="POST" action="/configs/{{.CustomerID}}/claim-resend" style="margin-top: 0.5rem;">
|
||||
{{.CSRFField}}
|
||||
<button type="submit" class="btn btn-outline btn-sm" data-confirm="Send a fresh code to the registered address? The previous code stops working immediately (the box activates it on its next report, ~15 min).">{{if .Claim.ClaimedAt}}Visszaállító kód küldése{{else}}Kód újraküldése{{end}}</button>
|
||||
<button type="submit" class="btn btn-outline btn-sm" data-confirm="Send a fresh code to the registered address? The previous code stops working immediately (the box activates it on its next report, ~15 min).">{{if .Claim.ClaimedAt}}Beállító kód küldése{{else}}Kód újraküldése{{end}}</button>
|
||||
</form>
|
||||
{{end}}
|
||||
|
||||
|
||||
@@ -317,6 +317,82 @@
|
||||
{{end}}
|
||||
</section>
|
||||
|
||||
<!-- Guest network (R-319): the R-54 watchdog's verdict per owned guest, and — the reason this
|
||||
card exists — HOW OFTEN IT HAD TO REPAIR EACH ONE. A guest the watchdog keeps fixing is
|
||||
healthy every time anyone looks and is nevertheless failing; showing only the state would
|
||||
give it a green tick, which is the failed-disk-drawn-as-a-healthy-empty-disk defect.
|
||||
|
||||
An unknown is NEVER drawn as healthy. Three absences, three sentences: an agent too old to
|
||||
report it, a capable agent that reported nothing, and a guest whose own state the watchdog
|
||||
did not assert. Each branch of this gate has its own render test (the template-gate rule).
|
||||
The agent has sent this on every heartbeat since v0.92.0 and nothing read it until now. -->
|
||||
<section class="card">
|
||||
<h2>Guest network
|
||||
{{if .GuestNet.AgentTooOld}}<span class="badge badge-neutral" title="This agent predates the guest-network watchdog">unknown</span>
|
||||
{{else if not .GuestNet.Reported}}<span class="badge badge-neutral" title="A capable agent sent no guest-network stanza">unknown</span>
|
||||
{{else if .GuestNet.Degraded}}<span class="badge badge-warn" title="A guest is unhealthy, or is being repaired repeatedly">needs attention</span>
|
||||
{{else if .GuestNet.UnknownCount}}<span class="badge badge-neutral" title="The watchdog did not assert a state for every guest">partly unknown</span>
|
||||
{{else if .GuestNet.Guests}}<span class="badge badge-ok" title="Every owned guest has an address, a default route and a live dhclient">healthy</span>
|
||||
{{end}}
|
||||
</h2>
|
||||
|
||||
{{if .GuestNet.AgentTooOld}}
|
||||
<p class="hint" style="color: var(--text-muted); font-size: 0.85rem;">
|
||||
This host's agent (<code>{{.AgentVersion}}</code>) does not run the guest-network watchdog —
|
||||
its guests' networking is <strong>unknown</strong>, not healthy. Needs agent <code>0.92.0</code> or newer.
|
||||
</p>
|
||||
{{else if not .GuestNet.Reported}}
|
||||
<p class="hint" style="color: var(--text-muted); font-size: 0.85rem;">
|
||||
This host reported no guest-network state, so it is <strong>unknown</strong> — not healthy.
|
||||
The watchdog is default-on; a silent capable agent means it was switched off
|
||||
(<code>guest_net.disable</code>), or this report predates it on this box.
|
||||
</p>
|
||||
{{else if .GuestNet.Guests}}
|
||||
<table class="data-table">
|
||||
<thead>
|
||||
<tr><th>Guest</th><th>State</th><th>Address</th><th>Route</th><th>dhclient</th><th>Repairs (1h)</th></tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{{range .GuestNet.Guests}}
|
||||
<tr>
|
||||
<td><code>{{.VMID}}</code></td>
|
||||
<td>
|
||||
{{if .Unknown}}<span class="badge badge-neutral" title="{{.Message}}">unknown</span>
|
||||
{{else if eq .State "healthy"}}<span class="badge badge-ok" title="{{.Message}}">healthy</span>
|
||||
{{else if eq .State "static_fault"}}<span class="badge badge-warn" title="{{.Message}}">static fault</span>
|
||||
{{else}}<span class="badge badge-error" title="{{.Message}}">unhealthy</span>{{end}}
|
||||
{{if .HealFailed}}<span class="badge badge-error" title="The watchdog TRIED to repair this guest and did not succeed">repair failed</span>{{end}}
|
||||
{{if .Damped}}<span class="badge badge-neutral" title="Repairs are rate-limited on this guest">damped</span>{{end}}
|
||||
</td>
|
||||
<td>{{if .IP}}<code>{{.IP}}</code> <span class="text-muted">({{.Mode}})</span>{{else}}<span class="text-muted">—</span>{{end}}</td>
|
||||
<td>{{if .HasRoute}}yes{{else}}<strong>no</strong>{{end}}</td>
|
||||
<td>{{if .DHClientAlive}}yes{{else if eq .Mode "static"}}<span class="text-muted">n/a</span>{{else}}<strong>no</strong>{{end}}</td>
|
||||
<td>
|
||||
{{if .Repairing}}<span class="badge badge-warn" title="The watchdog repaired this guest's network {{.RepairCount}} time(s) in the last hour — last at {{.LastHealAt}}">{{.RepairCount}}</span>
|
||||
{{else}}<span class="text-muted">0</span>{{end}}
|
||||
</td>
|
||||
</tr>
|
||||
{{end}}
|
||||
</tbody>
|
||||
</table>
|
||||
{{if .GuestNet.RepairingCount}}
|
||||
<p class="hint" style="font-size: 0.85rem; margin-top: 0.5rem;">
|
||||
<strong>{{.GuestNet.RepairingCount}} guest(s) had their networking repaired in the last hour.</strong>
|
||||
A guest that keeps being repaired reads healthy between repairs and is not. A killed
|
||||
<code>dhclient</code> once took a tunnel down for 1 h 15 m with nobody told; this row is that signal.
|
||||
</p>
|
||||
{{end}}
|
||||
<p class="hint" style="color: var(--text-muted); font-size: 0.85rem; margin-top: 0.5rem;">
|
||||
Last swept {{.GuestNet.CheckedAt}}. One row per owned <em>running</em> guest the watchdog has probed.
|
||||
</p>
|
||||
{{else}}
|
||||
<p class="hint" style="color: var(--text-muted); font-size: 0.85rem;">
|
||||
The watchdog ran and has no guest to report (swept {{.GuestNet.CheckedAt}}). This is
|
||||
distinguishable from silence: the stanza arrived, its guest list is empty.
|
||||
</p>
|
||||
{{end}}
|
||||
</section>
|
||||
|
||||
<!-- DR / Backup -->
|
||||
<section class="card">
|
||||
<h2>DR / Backup</h2>
|
||||
|
||||
@@ -103,13 +103,20 @@ GENERIC = {
|
||||
# someone must be able to re-check later, so none of them is bare. A quiet exclusion would be a
|
||||
# dropped field with paperwork, which is worse than the defect.
|
||||
#
|
||||
# TWO KINDS OF ENTRY, and the difference is deliberate:
|
||||
# THREE KINDS OF ENTRY, and the differences are deliberate:
|
||||
# * "redundant" — the hub already decodes something that answers the same question. No consumer
|
||||
# is wanted; the entry is the end of the matter.
|
||||
# * "R-264" — a fact with no consumer that ARGUABLY should have one. The entry does NOT
|
||||
# close the question; it records it against an OPEN register row so that
|
||||
# allowlisting cannot be mistaken for deciding. R-260 is closed by the gate plus
|
||||
# the operator-access fix; the leftover appetite is R-264.
|
||||
# * "not consumed, deliberately" — RULED by the operator, with the date. The question IS closed:
|
||||
# a reader was considered and declined on stated grounds. This kind was added on
|
||||
# 2026-08-13 because "arguably owed" had been carried for five days as though it
|
||||
# were a decision, and an undecided fact and a decided one must not read alike.
|
||||
# THE EMITTER IS DELIBERATELY LEFT ALONE: removing it is a coordinated two-repo
|
||||
# change and it also breaks the byte-identical cross-repo host-report golden. The
|
||||
# honest end here is a recorded decision, not a deletion.
|
||||
#
|
||||
# Keyed by (wire label, dotted emit path).
|
||||
_AH = "agent -> hub (POST /host-report)"
|
||||
@@ -123,6 +130,14 @@ _R264 = (
|
||||
"no consumer today, and one is arguably owed — recorded against R-264 (OPEN) rather than "
|
||||
"decided here. Allowlisting is not deciding.")
|
||||
|
||||
# RULED 2026-08-13 by the operator against R-264: these five were considered for a reader and
|
||||
# declined. The reason each is declined is per-entry below — a bare "not wanted" would be the quiet
|
||||
# exclusion this gate exists to prevent.
|
||||
_NOREADER = (
|
||||
"not consumed, DELIBERATELY — ruled 2026-08-13 (operator, against R-264). A reader was "
|
||||
"considered and declined; the question is closed, not open. The emitter stays (removing it is a "
|
||||
"two-repo change and breaks the host-report golden). ")
|
||||
|
||||
ALLOWLIST = {
|
||||
# ---- redundant: the hub already decodes an equivalent ----
|
||||
(_AH, "host.cpu_temp_c"): _REDUNDANT_HOST_METRICS,
|
||||
@@ -149,33 +164,54 @@ ALLOWLIST = {
|
||||
"redundant: hub-side wgsync reconciles peers from its own state, and the OOB path's own "
|
||||
"wg_handshake_age_s IS now decoded (into HostOOBRow, for the alert text)."),
|
||||
|
||||
# ---- no consumer, and one is arguably owed: R-264, OPEN ----
|
||||
(_AH, "guest_net"): _R264 + " The R-54 guest-network watchdog stanza (whole object).",
|
||||
(_AH, "guest_net.checked_at"): _R264,
|
||||
(_AH, "guest_net.guests.has_route"): _R264,
|
||||
(_AH, "guest_net.guests.dhclient_alive"): _R264,
|
||||
(_AH, "guest_net.guests.heal_succeeded"): _R264,
|
||||
(_AH, "guest_net.guests.heals_last_hour"): _R264,
|
||||
(_AH, "guest_net.guests.last_heal_at"): _R264,
|
||||
(_AH, "guest_net.guests.damped"): _R264,
|
||||
# ---- R-319: guest_net AND ALL SEVEN CHILDREN ARE GONE FROM THIS LIST ----
|
||||
# They are no longer allowlisted because they are no longer unconsumed: the hub models the R-54
|
||||
# watchdog stanza (web/hosts.go parseGuestNet → the host-detail Guest network card), including
|
||||
# heals_last_hour and heal_succeeded. Removing the entries is the POINT — an allowlisted tag is
|
||||
# SKIPPED by this gate, so leaving them here would mean the new reader's fields were never
|
||||
# actually checked for reachability, and the gate would report a coverage it did not have.
|
||||
# The first of R-264's readers; three groups remain owed one.
|
||||
|
||||
# ---- no consumer, and one is arguably owed: R-264, STILL OPEN ----
|
||||
(_AH, "selfupdate_pending"): _R264 + (
|
||||
" NOTE: the agent's own comment beside this field claimed 'the hub reads an absent field as "
|
||||
"pending=false, the correct default'. The hub had no field at all, so it read nothing "
|
||||
"either way. The comment was corrected in the same change as this entry."),
|
||||
(_AH, "selfupdate_pending_version"): _R264,
|
||||
(_AH, "mgmt_plane.healed_recently"): _R264 + (
|
||||
" The hub DOES alarm on mgmt_plane.privsep_healed_at, which is the timestamp beside this "
|
||||
"boolean, so the recurring-clobber signal is not lost — only this flag is."),
|
||||
(_AH, "pbs_dr.applied_at"): _R264,
|
||||
(_AH, "mgmt_plane.healed_recently"): _NOREADER + (
|
||||
"The recurring-clobber signal is NOT lost: the hub already alarms on "
|
||||
"mgmt_plane.privsep_healed_at, the timestamp sitting beside this boolean. A second reader "
|
||||
"for the flag would add a second way to say the same thing and no new fact."),
|
||||
(_AH, "pbs_dr.applied_at"): _NOREADER + (
|
||||
"A tier-applied TIMESTAMP, and the hub already decodes pbs_dr.state — which is the verdict. "
|
||||
"Presence is not success: consuming the timestamp without the state is precisely the "
|
||||
"attempt-read-as-result trap, and with the state it answers nothing further."),
|
||||
(_AH, "restore_tests.mount_parity"): _R264 + (
|
||||
" R-262: the hub's own comment claims this contract is mirrored field-for-field and that a "
|
||||
"key-set test guards drift; it is two fields short and the fixture omits the same two."),
|
||||
(_AH, "restore_tests.mount_inventory"): _R264 + " R-262, as mount_parity.",
|
||||
(_CH, "config_hash"): _R264,
|
||||
(_CH, "reporting_disabled"): _R264,
|
||||
(_CH, "stacks"): _R264 + (
|
||||
" The whole per-stack report object; the hub's app view is built from app_telemetry."),
|
||||
(_CH, "storage.migrated_to"): _R264,
|
||||
(_CH, "config_hash"): _NOREADER + (
|
||||
"A config FINGERPRINT. The hub authors the config and knows its own generation "
|
||||
"(desired_generation, config_version), which is the convergence question it actually asks. "
|
||||
"A hash it did not compute answers a question nobody has posed."),
|
||||
(_CH, "reporting_disabled"): (
|
||||
"redundant: the box sends health.status = \"disabled\" in the SAME minimal report "
|
||||
"(controller cmd/controller/main.go:1254), the hub decodes it into reports.health_status, and "
|
||||
"web/rollup.go:25 renders that customer as 'disabled'. The state IS visible; this flag is a "
|
||||
"second spelling of a fact already read. ⚠ DECIDED ON ITS OWN MERITS 2026-08-13, and the "
|
||||
"decision came with a REAL finding that a second decoded flag would NOT have fixed: "
|
||||
"StalenessChecker.Check (monitor/staleness.go:88+) is age-only — it skips BLOCKED customers "
|
||||
"and nothing else — so a deliberately-silent box still goes stale at 30 min and down at 60. "
|
||||
"Filed as R-321. The fix belongs in the checker, which already has the health status it "
|
||||
"needs; adding a field here would have felt like progress and left the alarm firing."),
|
||||
(_CH, "stacks"): _NOREADER + (
|
||||
"The whole per-stack report object. The hub's app view is built from app_telemetry, which is "
|
||||
"a purpose-built wire with its own table; a second, differently-shaped source for the same "
|
||||
"screen is how two answers to one question get shipped."),
|
||||
(_CH, "storage.migrated_to"): _NOREADER + (
|
||||
"A drive-migration marker: box-local bookkeeping about where data was moved ON that box. The "
|
||||
"hub holds no drive-layout intent to reconcile it against, so it could only be displayed, "
|
||||
"and a fact displayed with nothing to compare it to is decoration."),
|
||||
(_CH, "backup.last_db_dump"): _R264,
|
||||
(_CH, "backup.last_integrity_check"): _R264,
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user