felhom.eu/documentation/operations/tailscale.md

Tailscale — location-independent access to the demo nodes (N100 + HP t740)

Added 2026-07-19. Goal: the demo Proxmox host (N100, felhom-pve) can be taken to a different LAN (e.g. a vacation home) and stay reachable from DooPlex, while its own dependencies keep working. The ssh felhom-pve alias remains the interface — only its transport changed.

Topology (who is what)

Node	Tailnet name	Tailnet IPv4	Transport it provides	Managed as
DooPlex (192.168.0.180)	dooplex	100.107.87.53	subnet router advertising 192.168.0.0/24; expiry disabled	k3s pod admin-system/tailscale (hostNetwork, GitOps/ArgoCD) — 157 d old, predates this task
N100 (192.168.0.162)	felhom-pve	100.70.170.35	its own node; expiry disabled	host package tailscale (Debian trixie apt repo), tailscaled.service
HP t740 (demo-hp-bb76ea)	demo-hp	100.76.96.79	its own node; expiry NOT yet disabled — see below	host package tailscale 1.98.9 (Debian trixie apt repo), tailscaled.service, added 2026-07-21

Tailnet: nagyfenyvesi.viktor@gmail.com (Tailscale Free). Both nodes run v1.98.x.

ssh felhom-pve now resolves to 100.70.170.35 (see ~/.ssh/config on DooPlex). Tailscale takes the direct LAN path when both boxes are home (measured: pong … via 192.168.0.162:41641 in 1ms) and falls back to the tunnel / a DERP relay when the N100 is remote. felhom-pve-lan → 192.168.0.162 is the explicit LAN-only fallback.

This is an OPERATOR-LAB EXCEPTION, not product shape

Real customer boxes never get tailscale. Customer operator-access is the WireGuard tunnel plus the H1 OOB path, full stop. Tailscale exists on felhom-pve and demo-hp only because those two are lab machines that live on LANs the operator does not control.

demo-hp is the confusing one: it is customer-shaped — a normal appliance install, a real customer record, a real guest, a real day-0 — with tailscale bolted on afterwards by hand. A future product-shape audit that finds it there must not conclude the product ships it. See operations/nodes.md.

demo-hp joined 2026-07-21

tailscale up --authkey=file:… --hostname=demo-hp --accept-routes=false, then tailscale set --accept-dns=false (see the hard rule below — it was needed, and why is worth reading). Auth was a pre-auth key, passed as --authkey=file:<path> and shredded immediately so it never appeared in the box's process list. Verified from DooPlex: ping ~40 ms, direct SSH with no ProxyJump, and the path is direct (37.191.56.193:45127), not a DERP relay.

OPEN: key expiry is still enabled on demo-hp (expires 2027-01-17), unlike the other two nodes which have it disabled. Disabling it is a per-device admin-console toggle (Machines → demo-hp → Disable key expiry) or an API call with a tskey-api-… token; a tskey-auth-… pre-auth key cannot do it (verified: the API returns 401). Until it is toggled the node will drop off the tailnet on that date and need re-authentication.

Hard rule: --accept-dns=false on the host node

The N100's tailscaled runs with --accept-dns=false (CorpDNS:false). MagicDNS must never rewrite /etc/resolv.conf: the box relies on pi-hole (192.168.0.250) for split-horizon (gitea.dooplex.hu → 192.168.0.180). Verified byte-identical after tailscale up.

On demo-hp this rule was broken and then fixed, which is the useful part of the story. The join omitted --accept-dns=false, and MagicDNS promptly rewrote /etc/resolv.conf to nameserver 100.100.100.100 — exactly what this rule forbids. Nothing broke at the vacation site, because there is no pi-hole there and both gitea.dooplex.hu and hub.felhom.eu resolve publicly anyway; the damage would have appeared silently when the box comes home, where split-horizon is what makes gitea.dooplex.hu → 192.168.0.180 local. tailscale set --accept-dns=false restored nameserver 192.168.0.1 immediately, with the tailnet and the agent unaffected. Pass --accept-dns=false at join time on any host node; do not rely on noticing afterwards. (DooPlex's containerised node runs TS_ACCEPT_DNS=true, but because it is a pod with its own mount namespace that only rewrites the container's resolv.conf — the DooPlex host resolv.conf is untouched.)

The accept-routes spike (measured 2026-07-19) — leave it OFF at home

--accept-routes is OFF on felhom-pve and should stay off while the box is on 192.168.0.0/24. Enabling it there was tested and degraded local traffic:

• Tailscale installed 192.168.0.0/24 dev tailscale0 into routing table 52 with an ip rule 5270: from all lookup 52 that outranks main, so the whole local subnet hijacked onto the tunnel: ip route get 192.168.0.180 and …0.250 flipped from vmbr0 to tailscale0, ping RTT to .180 went 0.3 ms → ~68 ms, and inbound SSH went asymmetric and stalled.
• PBS was unaffected (10.77.0.1 stays on wg-felhom), because PBS does not use this subnet.
• Rollback (tailscale set --accept-routes=false) restored direct vmbr0 immediately.

When travelling, the N100 is on a different subnet, so accepting 192.168.0.0/24 no longer overlaps the local net and is safe — it lets the box reach 192.168.0.180/gitea over the tunnel. Treat it as an opt-in, travel-only toggle you turn on only if you need a home-LAN service (e.g. a new controller-image pull), and turn back OFF before the box returns home. Management SSH over 100.x is immune to the hairpin either way (that path was used to perform the rollback).

PBS does NOT depend on the home LAN

Contrary to the original assumption, PBS is offsite: storage.cfg targets 10.77.0.1 over the WireGuard interface wg-felhom whose endpoint is a public Hetzner address (…:443). It works from any location with internet and needs neither the subnet router nor --accept-routes. This is why the subnet router is nice-to-have (gitea image pulls while away), not load-bearing.

Key expiry

Disabled on both nodes in the admin console (a node whose key expires mid-trip drops off the tailnet). Confirm on the Machines page: each row shows an Expiry disabled badge.

Rollback / uninstall (host node)

ssh felhom-pve "tailscale set --accept-routes=false"     # undo the travel toggle
ssh felhom-pve "tailscale down"                           # leave the tailnet (reversible: tailscale up)
ssh felhom-pve "systemctl disable --now tailscaled && apt-get purge -y tailscale"   # full removal
# then repoint ~/.ssh/config Host felhom-pve back to 192.168.0.162 (or use felhom-pve-lan)

Portability of the N100 itself

• Host network is DHCP (converted 2026-07-19; iface vmbr0 inet dhcp). At home a DHCP reservation pins MAC 68:1d:ef:5d:a6:64 → 192.168.0.162 so home behaviour is unchanged. Revert on the box: cp /root/interfaces.static-revert-20260719 /etc/network/interfaces && ifreload -a.
• Guest 9201 is already DHCP.

⚠️ Known blocker — the controller↔agent plane is pinned to 192.168.0.162 (NOT fixed)

felhom-agent binds listen_addr: 192.168.0.162:8443 (in /etc/felhom-agent/agent.json) and guest 9201's /etc/felhom-bootstrap/bootstrap.json points its controller at 192.168.0.162:8443. On a foreign LAN the host will not have 192.168.0.162, so the agent cannot bind it (agent down) and the guest cannot reach it. Tailscale and host-DHCP do not fix this (guests bridge at L2). Making the box fully functional off-LAN needs a design change — bind 0.0.0.0:8443 and give the guest a stable way to reach the host (e.g. a host-only bridge with a fixed private IP). Until then the N100 is reachable and backs up off-LAN, but its local operator plane is not portable.

Off-LAN DNS note

At a foreign LAN, DHCP repoints the host's resolv.conf at the local resolver, so gitea.dooplex.hu (pi-hole split-horizon) will not resolve. To pull a controller image while away, either turn on the travel --accept-routes toggle and add a temporary 192.168.0.180 gitea.dooplex.hu line to /etc/hosts, or point DNS at 192.168.0.250 over the subnet route. External customer access is via Cloudflare Tunnel and is unaffected.

Vacation-day checklist (for the operator)

1. At the destination, connect the N100 by wired ethernet, power on, wait ~2 min.
2. From DooPlex: tailscale-side — check the pod: sudo kubectl -n admin-system exec deploy/tailscale -- tailscale status shows felhom-pve online (likely relayed/DERP at first — fine).
3. ssh felhom-pve pveversion — management reachable over the tailnet.
4. ssh felhom-pve "pvesm status | grep -i pbs" — the real off-LAN PBS test (could not be validated at home; PBS rides its own WireGuard tunnel, so it should be green).
5. Demo URL via Cloudflare Tunnel loads (customer-facing path, independent of all the above).
6. If you need a home-LAN service (gitea image pull): ssh felhom-pve "tailscale set --accept-routes=true"
   • /etc/hosts entry as above — then set it back to false before the box comes home.
7. Expect the controller↔agent plane to be down off-LAN until the pinned-.162 blocker above is fixed.