admin/felhom.eu
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: node inventory for the two-host fleet; demo-hp on the tailnet

Browse Source 
New documentation/operations/nodes.md: HP t740 hardware, disks (SanDisk system
SSD 182195804614; Toshiba 1TB NVMe 58BS11AFT8MQ PRESENT AND UNENROLLED, still
NTFS, do not touch), the five-NIC map and the trap that cost the first install,
and the access path - no SSH key is baked, auth is the hub-vaulted G1
break-glass password.

tailscale.md gains demo-hp, the operator-lab-exception warning so a future
product-shape audit does not conclude the product ships tailscale, and the
--accept-dns evidence: the join omitted the flag, MagicDNS rewrote
/etc/resolv.conf, and it was reverted. Harmless at the vacation site, would
have bitten silently at home where split-horizon matters.

OPEN: key expiry still enabled on demo-hp (2027-01-17) - needs an admin-console
toggle or an API token; a pre-auth key cannot do it.
This commit is contained in:
authentik Default Admin
2026-07-21 18:51:05 +02:00
parent 460cc73bad
commit 8ec45f3b42
3 changed files with 158 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CONTEXT.md
+17
View File
@@ -3,6 +3,23 @@
> Created with the REUSE.md rollout (2026-07-03). Authoritative history: `hub/CHANGELOG.md` (hub),
> `website/CHANGELOG.md`, `scripts/CHANGELOG.md`; end-of-task detail in `REPORT.md`.
- **2026-07-21 — THE FLEET IS TWO HOSTS.** A second Proxmox node exists: **`demo-hp-bb76ea`**, an
**HP t740 Thin Client** (Ryzen V1756B, 30 GiB, PVE node `felhom-host`, customer `demo-hp`), installed
from the armed universal ISO the same day. Both hosts run **agent 0.92.1**; the N100 is
`demo-felhom-8363b5`. **Full inventory: `documentation/operations/nodes.md`** — read it before
touching the HP, because two things there are load-bearing: the box's **1TB Toshiba NVMe is present
and UNENROLLED** (still carrying its old NTFS partition; a future Tárhely candidate that must join
through the storage flow, never the installer), and there is **no operator SSH key baked** on it, so
access is the hub-vaulted G1 break-glass password (R-61 is the fix for that lockout).
**Both nodes are at the VACATION site and travel home ~2026-08-02.** `demo-hp` joined the tailnet as
`100.76.96.79` — an **operator-lab exception**, not product shape; real customer boxes never get
tailscale. **Second-hardware proof, worth keeping:** the pairing/day-0 chain ran end to end on virgin
hardware it had never seen, the **shim loader booted with Secure Boot ENABLED** (so SB-off was an
N100-firmware workaround, not a Felhom requirement), and the box **self-lifted the floor
0.153.0 → 0.156.0 during day-0**. The install was not clean, and the failures are filed: **R-59**
(no DHCP on the 4-port NIC → the installer baked a static fallback instead of aborting), **R-60**
(first-boot NIC sweep self-heal), **R-61** (baked root password unknowable).
- **2026-07-19 — N100 MADE LOCATION-INDEPENDENT via Tailscale.** `felhom-pve` (the N100) joined the
tailnet as a host package (`100.70.170.35`, expiry disabled); DooPlex already advertised
`192.168.0.0/24` via its GitOps k3s `admin-system/tailscale` pod (`100.107.87.53`). `ssh felhom-pve`
documentation/operations/nodes.md
+104
View File
@@ -0,0 +1,104 @@
# Fleet node inventory — the physical demo/lab hosts
> Added 2026-07-21, when the fleet stopped being one box. Two Proxmox hosts now check in to the hub.
> This is the operator-facing inventory: what the hardware is, how to reach it, and what is
> deliberately NOT enrolled on it.
## The fleet
| | `demo-felhom-8363b5` | `demo-hp-bb76ea` |
|---|---|---|
| Hardware | N100 mini-PC | **HP t740 Thin Client** (s/n `8CN944035T`) |
| CPU / RAM | Intel N100 | **AMD Ryzen Embedded V1756B**, 8 threads / **30 GiB** |
| Firmware | AMI AN3PLUS-class | **AMI M42 v01.10 (11/11/2020)** |
| PVE node name | `demo-felhom` | `felhom-host` |
| Customer | `demo-felhom` | `demo-hp` |
| Agent | 0.92.1 | 0.92.1 |
| SSH alias | `felhom-pve` | **`demo-hp`** |
| Tailnet | `100.70.170.35` | **`100.76.96.79`** |
| Loader used to install | `mkimage` (unsigned, SB **off** — firmware workaround) | **`shim`, Secure Boot ENABLED** |
**Both are at the VACATION site** and travel home with the rest of the kit **~2026-08-02**. While
away, their LAN addresses are on that site's `192.168.0.0/24`: N100 `.147`/`.162`, HP `.87`. The
tailnet addresses are the stable ones — use those.
## demo-hp — the HP t740, in detail
### Disks
| device | model | serial | role |
|---|---|---|---|
| `sda` | SanDisk X600 M.2 2280 SATA 128GB | `182195804614` | **system disk** — PVE, LVM (`pve-root` 39.6G, `pve-data` thin pool, guest 9201's three volumes) |
| `nvme0n1` | KXG50PNV1T02 NVMe TOSHIBA 1024GB | `58BS11AFT8MQ` | **PRESENT AND UNENROLLED — do not touch** |
> **The NVMe is a future Tárhely candidate and is deliberately outside everything.** It still holds
> its previous **NTFS** partition, is not mounted, and appears in no LVM PV and no ZFS pool. The
> install ISO's exact-serial filter pinned `sda` only, which is why it survived. **It joins the
> product later through the normal Tárhely (storage) flow — never through the installer, and never
> by hand.** Anything that formats it before that decision is a mistake.
### NIC map — and the trap
This board has **five wired interfaces**, and the obvious one is the wrong one:
| interface | MAC | driver | what it is | state at this site |
|---|---|---|---|---|
| `enp1s0f0` | `a0:36:9f:5d:07:20` | `igb` | 4-port expansion card | no carrier, **no DHCP** |
| `enp1s0f1` | `a0:36:9f:5d:07:21` | `igb` | ″ | no carrier |
| `enp1s0f2` | `a0:36:9f:5d:07:22` | `igb` | ″ | no carrier |
| `enp1s0f3` | `a0:36:9f:5d:07:23` | `igb` | ″ | no carrier |
| **`enp2s0f0`** | `7c:d3:0a:77:d9:76` | `r8169` | **onboard port — the one that works** | carrier up, 1000 Mb, **this is `vmbr0`'s port** |
| `wlo1` | `24:ee:9a:e5:05:b0` | `iwlwifi` | wifi | unused |
**This trap cost the first install.** The 4-port card got no lease, and instead of aborting the
installer baked its `192.168.100.2` fallback as a **static** `vmbr0` address and completed — a box
that looked installed and could never call home. Repaired on the console by bridging `vmbr0` to
`enp2s0f0`. Filed as **R-59** (must hard-abort) and **R-60** (first-boot NIC sweep self-heal).
Current, post-repair: `vmbr0` static `192.168.0.87/24`, gw `192.168.0.1`, bridge-port `enp2s0f0`.
No trace of `192.168.100.2` remains. `wg-felhom` `10.77.0.3/32` up to the hub. Guest **9201
`demo-hp`** running.
### Access — there is no baked SSH key
`ssh demo-hp` resolves to the tailnet address, but **no operator public key is on this box** — the
HP profile deliberately left `FELHOM_ROOT_SSH_KEY` blank. Authentication is the **G1 break-glass
root password vaulted in the hub**, `host_recovery` row `demo-hp-bb76ea` (set at day-0,
2026-07-21 16:24 UTC).
Retrieval (operator-side, and **shred the copy** — that DB holds every host's secret):
```bash
sudo kubectl -n felhom-system exec <hub-pod> -- cat /data/hub.db > /tmp/x.db
python3 -c "import sqlite3;print(sqlite3.connect('/tmp/x.db').execute(
\"SELECT secret FROM host_recovery WHERE host_id='demo-hp-bb76ea'\").fetchone()[0])"
shred -u /tmp/x.db
```
Then `sshpass -e ssh root@demo-hp` (sshpass is on DooPlex, not on the nodes).
**This is the lockout filed as R-61**: the ISO mints a throwaway root password per build and discards
the plaintext, so the console is unreachable without a working hub and network — precisely what you
may be trying to fix. Slice 1 is to emit the baked password into the build report.
`demo-hp-lan` (`192.168.0.87` via `ProxyJump felhom-pve`) is the fallback while the box is away.
## Tailscale on demo-hp is an OPERATOR-LAB EXCEPTION
> **Read this before any product-shape audit.** `demo-hp` is **customer-shaped** — it is a normal
> appliance install with a real customer record (`demo-hp`), a real guest, and a real day-0.
> **Tailscale is not part of that shape.** It was installed by hand on 2026-07-21 purely so the
> operator can reach a lab box that lives on someone else's LAN.
>
> **Real customer boxes never get tailscale.** Their operator access is the WireGuard tunnel plus
> the H1 OOB path, and nothing else. If a future audit finds tailscale on `demo-hp` and concludes
> the product ships it — that conclusion is wrong, and this paragraph is the reason. The same
> exception already applies to `felhom-pve`.
Details, and the two hard rules that apply to any host node, in `operations/tailscale.md`.
## What is NOT enrolled here (deliberately)
- The **1TB NVMe** on demo-hp — see above.
- **No PBS datastore, no offsite target** on demo-hp yet; the DR tier is the N100's.
- **No second customer guest** beyond 9201.
documentation/operations/tailscale.md
+37 -2
View File
@@ -1,4 +1,4 @@
# Tailscale — location-independent access to felhom-pve (the N100)
# Tailscale — location-independent access to the demo nodes (N100 + HP t740)
> Added 2026-07-19. Goal: the demo Proxmox host (N100, `felhom-pve`) can be taken to a different
> LAN (e.g. a vacation home) and stay reachable from DooPlex, while its own dependencies keep
@@ -10,6 +10,7 @@
|---|---|---|---|---|
| DooPlex (192.168.0.180) | `dooplex` | `100.107.87.53` | **subnet router** advertising `192.168.0.0/24`; expiry disabled | **k3s pod** `admin-system/tailscale` (`hostNetwork`, GitOps/ArgoCD) — 157 d old, predates this task |
| N100 (192.168.0.162) | `felhom-pve` | `100.70.170.35` | its own node; expiry disabled | **host package** `tailscale` (Debian trixie apt repo), `tailscaled.service` |
| HP t740 (`demo-hp-bb76ea`) | `demo-hp` | `100.76.96.79` | its own node; **expiry NOT yet disabled — see below** | **host package** `tailscale` 1.98.9 (Debian trixie apt repo), `tailscaled.service`, added 2026-07-21 |
Tailnet: `nagyfenyvesi.viktor@gmail.com` (Tailscale Free). Both nodes run v1.98.x.
@@ -18,11 +19,45 @@ the **direct LAN path** when both boxes are home (measured: `pong … via 192.16
and falls back to the tunnel / a DERP relay when the N100 is remote. `felhom-pve-lan` →
`192.168.0.162` is the explicit LAN-only fallback.
## This is an OPERATOR-LAB EXCEPTION, not product shape
**Real customer boxes never get tailscale.** Customer operator-access is the WireGuard tunnel plus
the H1 OOB path, full stop. Tailscale exists on `felhom-pve` and `demo-hp` only because those two are
lab machines that live on LANs the operator does not control.
`demo-hp` is the confusing one: it is **customer-shaped** — a normal appliance install, a real
customer record, a real guest, a real day-0 — with tailscale bolted on afterwards by hand. A future
product-shape audit that finds it there must not conclude the product ships it. See
`operations/nodes.md`.
## demo-hp joined 2026-07-21
`tailscale up --authkey=file:… --hostname=demo-hp --accept-routes=false`, then
`tailscale set --accept-dns=false` (see the hard rule below — it was needed, and why is worth
reading). Auth was a pre-auth key, passed as `--authkey=file:<path>` and shredded immediately so it
never appeared in the box's process list. Verified from DooPlex: `ping` ~40 ms, direct SSH with no
ProxyJump, and the path is **direct** (`37.191.56.193:45127`), not a DERP relay.
**OPEN: key expiry is still enabled on `demo-hp`** (expires `2027-01-17`), unlike the other two
nodes which have it disabled. Disabling it is a per-device **admin-console toggle** (Machines →
demo-hp → Disable key expiry) or an API call with a `tskey-api-…` token; a `tskey-auth-…` pre-auth
key cannot do it (verified: the API returns 401). Until it is toggled the node will drop off the
tailnet on that date and need re-authentication.
## Hard rule: `--accept-dns=false` on the host node
The N100's `tailscaled` runs with **`--accept-dns=false`** (`CorpDNS:false`). MagicDNS must never
rewrite `/etc/resolv.conf`: the box relies on pi-hole (`192.168.0.250`) for split-horizon
(`gitea.dooplex.hu` → `192.168.0.180`). Verified byte-identical after `tailscale up`. (DooPlex's
(`gitea.dooplex.hu` → `192.168.0.180`). Verified byte-identical after `tailscale up`.
**On `demo-hp` this rule was broken and then fixed, which is the useful part of the story.** The join
omitted `--accept-dns=false`, and MagicDNS promptly rewrote `/etc/resolv.conf` to
`nameserver 100.100.100.100` — exactly what this rule forbids. Nothing broke *at the vacation site*,
because there is no pi-hole there and both `gitea.dooplex.hu` and `hub.felhom.eu` resolve publicly
anyway; the damage would have appeared silently **when the box comes home**, where split-horizon is
what makes `gitea.dooplex.hu` → `192.168.0.180` local. `tailscale set --accept-dns=false` restored
`nameserver 192.168.0.1` immediately, with the tailnet and the agent unaffected. **Pass
`--accept-dns=false` at join time on any host node; do not rely on noticing afterwards.** (DooPlex's
containerised node runs `TS_ACCEPT_DNS=true`, but because it is a pod with its own mount namespace
that only rewrites the *container's* resolv.conf — the DooPlex host resolv.conf is untouched.)