Files
obsidian-vault/family/how-to/rasputin-router.md
T

700 lines
18 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: Rasputin OpenWrt router
created: '2026-07-22'
updated: '2026-09-15'
tags:
- openwrt
- router
- rasputin
- networking
- vless
- cloudflared
---
# Rasputin R5S OpenWrt router
> Status as of 2026-07-22: Rasputin is the OpenWrt edge/LAN router, routing normal LAN traffic through a VLESS/Xray tunnel via `tun0`, with explicit direct-WAN exceptions for upstream LAN access and Mac `cloudflared` QUIC traffic.
## Identity
| Field | Value |
|---|---|
| Hostname | `Rasputin` |
| Hardware | FriendlyElec NanoPi R5S |
| Board | `friendlyarm,nanopi-r5s` |
| CPU | ARMv8 Processor rev 0 |
| OpenWrt | `24.10.2`, revision `r28739-d9340319c6` |
| Target | `rockchip/armv8` |
| Kernel | `6.6.93` |
| Rootfs | squashfs |
| LAN IP | `192.168.6.1/24` |
| IPv6 ULA prefix | `fd3b:153e:9be9::/48` |
## Network Topology
```text
LAN clients
-> br-lan / 192.168.6.0/24
-> policy routing
-> most LAN traffic: table vpn -> tun0 -> badvpn-tun2socks -> Xray SOCKS -> VLESS
-> selected exceptions: table main -> WAN device
```
WAN is currently represented by both `eth0` and `usb0` in the `wan` firewall zone. At capture time, `eth0` had no carrier, while `usb0` was up and had the default route.
Current default route:
```text
default via 10.252.240.133 dev usb0 src 10.252.240.230
```
VPN table:
```text
default dev tun0 scope link
```
## Interfaces
### LAN
`br-lan` bridges:
```text
eth1
eth2
phy0-ap0
```
LAN config:
```text
network.lan=interface
network.lan.device='br-lan'
network.lan.proto='static'
network.lan.ipaddr='192.168.6.1'
network.lan.netmask='255.255.255.0'
network.lan.ip6assign='60'
network.lan.dns='8.8.8.8'
```
### WAN
`eth0`:
```text
network.wan=interface
network.wan.device='eth0'
network.wan.proto='dhcp'
network.wan.peerdns='0'
network.wan.type='bridge'
```
`usb0`:
```text
network.usbwan=interface
network.usbwan.proto='dhcp'
network.usbwan.device='usb0'
```
Firewall `wan` zone includes:
```text
wan
wan6
wwan
usbwan
```
### VPN
`tun0` is declared as an unmanaged interface:
```text
network.vpn=interface
network.vpn.proto='none'
network.vpn.device='tun0'
```
`tun0` is created/used by `badvpn-tun2socks`, launched from `/etc/init.d/tunvpn`.
## DHCP And Static Leases
DHCP server:
```text
dhcp.lan=dhcp
dhcp.lan.interface='lan'
dhcp.lan.start='100'
dhcp.lan.limit='150'
dhcp.lan.leasetime='12h'
dhcp.lan.dhcp_option='6,1.1.1.1,1.0.0.1'
dhcp.lan.dhcpv4='server'
dhcp.lan.dhcpv6='server'
dhcp.lan.ra='server'
dhcp.lan.ra_slaac='1'
dhcp.lan.ra_flags='managed-config' 'other-config'
```
Static DHCP reservations added 2026-07-14 for the Mac:
```text
dhcp.@host[0].name='mbp-black-eth'
dhcp.@host[0].mac='00:e0:4c:68:04:88'
dhcp.@host[0].ip='192.168.6.104'
dhcp.@host[1].name='mbp-black-wifi'
dhcp.@host[1].mac='66:02:b1:8b:2f:94'
dhcp.@host[1].ip='192.168.6.173'
```
Backup made before this change:
```text
/etc/config/dhcp.bak.20260714-195408
```
## VLESS / Xray Stack
Packages present include:
```text
badvpn
xray-core
v2ray-core
v2ray-extra
v2ray-geoip
v2ray-geosite
v2raya
luci-app-v2raya
sing-box
nftables-json
iptables-nft
```
Runtime processes:
```text
badvpn-tun2socks --tundev tun0 --netif-ipaddr 10.99.0.2 --netif-netmask 255.255.255.252 --socks-server-addr 127.0.0.1:1080 --loglevel none
/usr/bin/xray run -confdir /etc/xray -config /etc/xray/config.json -format json
```
`v2raya` is installed, but the observed UCI service config previously had `enabled='0'`. The active traffic path is the custom `tunvpn` service plus direct Xray config.
### Xray Config
File:
```text
/etc/xray/config.json
```
Structure:
```json
{
"log": {
"loglevel": "debug",
"access": "/tmp/xray-access.log",
"error": "/tmp/xray-error.log"
},
"inbounds": [
{
"tag": "socks-in",
"listen": "127.0.0.1",
"port": 1080,
"protocol": "socks"
}
],
"outbounds": [
{
"tag": "proxy",
"protocol": "vless",
"settings": {
"vnext": [
{
"address": "188.239.191.235",
"port": 443,
"users": [
{
"id": "<redacted-vless-user-id>",
"flow": "xtls-rprx-vision",
"encryption": "none"
}
]
}
]
},
"streamSettings": {
"network": "tcp",
"security": "reality",
"realitySettings": {
"serverName": "node3.sysnx.net",
"publicKey": "<redacted-reality-public-key>",
"shortId": "<redacted-reality-short-id>",
"fingerprint": "qq"
}
}
}
],
"routing": {
"rules": [
{ "type": "field", "outboundTag": "proxy", "network": "tcp,udp" }
]
}
}
```
Secrets are intentionally redacted in this note. The live file on the router contains the actual VLESS UUID and REALITY parameters.
## `tunvpn` Service
File:
```text
/etc/init.d/tunvpn
```
Purpose:
- Start `badvpn-tun2socks`.
- Populate routing table `vpn` with default route through `tun0`.
- Add policy rules so LAN traffic uses the VPN table.
- Add explicit Cloudflare Tunnel QUIC bypass rules for the Mac.
Current script:
```sh
#!/bin/sh /etc/rc.common
START=95
STOP=10
start() {
badvpn-tun2socks --tundev tun0 --netif-ipaddr 10.99.0.2 --netif-netmask 255.255.255.252 --socks-server-addr 127.0.0.1:1080 --loglevel none &
sleep 2
ip route replace default dev tun0 table vpn
# cloudflared QUIC direct bypass for Mac Ethernet/Wi-Fi
ip rule add priority 98 from 192.168.6.173 ipproto udp dport 7844 lookup main 2>/dev/null
ip rule add priority 99 from 192.168.6.104 ipproto udp dport 7844 lookup main 2>/dev/null
ip rule add iif br-lan lookup vpn priority 101
}
stop() {
killall badvpn-tun2socks
ip rule del priority 98 2>/dev/null
ip rule del priority 99 2>/dev/null
ip rule del iif br-lan lookup vpn 2>/dev/null
ip route flush table vpn
}
```
Backup made before the Cloudflare direct-bypass update:
```text
/etc/init.d/tunvpn.bak.20260714-195530
```
## Policy Routing
Current rules:
```text
0: from all lookup local
98: from 192.168.6.173 ipproto udp dport 7844 lookup main
99: from 192.168.6.104 ipproto udp dport 7844 lookup main
100: from all to 192.168.2.0/24 lookup main
101: from all iif br-lan lookup vpn
32766: from all lookup main
32767: from all lookup default
```
Meaning:
- Mac Wi-Fi `192.168.6.173` UDP/7844 goes direct through `main`.
- Mac Ethernet `192.168.6.104` UDP/7844 goes direct through `main`.
- Traffic to upstream LAN `192.168.2.0/24` goes direct through `main`.
- All other traffic entering from LAN bridge `br-lan` uses `table vpn`.
- `table vpn` sends default traffic through `tun0`.
`/etc/rc.local` adds the upstream-LAN bypass:
```sh
# Put your custom commands here that should be executed once
# the system init finished. By default this file does nothing.
ip rule add to 192.168.2.0/24 lookup main priority 100
exit 0
```
## Firewall Model
Default firewall policy:
```text
input: REJECT
output: ACCEPT
forward: REJECT
```
Zones:
### `lan`
```text
name='lan'
input='ACCEPT'
output='ACCEPT'
forward='REJECT'
masq='1'
network='lan'
```
### `wan`
```text
name='wan'
input='REJECT'
output='ACCEPT'
forward='REJECT'
masq='1'
mtu_fix='1'
network='wan' 'wan6' 'wwan' 'usbwan'
```
### `vpn`
```text
name='vpn'
input='ACCEPT'
output='ACCEPT'
forward='ACCEPT'
masq='1'
network='vpn'
```
Forwarding:
```text
lan -> vpn
```
There is no general `lan -> wan` forwarding. Instead, selected `lan -> wan` traffic is allowed before a LAN-to-WAN killswitch.
## Important Firewall Rules
### Cloudflare Tunnel QUIC Direct Bypass
Added 2026-07-14:
```text
firewall.@rule[8].name='Allow-Mac-cloudflared-QUIC-direct'
firewall.@rule[8].src='lan'
firewall.@rule[8].dest='wan'
firewall.@rule[8].src_ip='192.168.6.104' '192.168.6.173'
firewall.@rule[8].proto='udp'
firewall.@rule[8].dest_port='7844'
firewall.@rule[8].target='ACCEPT'
```
The nft realization appears in `forward_lan` before the killswitch:
```text
ip saddr { 192.168.6.104, 192.168.6.173 } udp dport 7844 jump accept_to_wan comment "!fw4: Allow-Mac-cloudflared-QUIC-direct"
```
At capture time, this rule had matched traffic:
```text
counter packets 19 bytes 27636
```
This confirms that Mac `cloudflared` QUIC traffic was hitting the direct-WAN exception.
### Upstream LAN Bypass (`Allow-upstream-LAN`)
```text
firewall.@rule[10].name='Allow-upstream-LAN'
firewall.@rule[10].src='lan'
firewall.@rule[10].dest='wan'
firewall.@rule[10].target='ACCEPT'
firewall.@rule[10].dest_ip='192.168.2.0/24' '192.168.0.1'
```
---
## 🔴 ICMP к недоступному хосту в 192.168.2.0/24 отдаёт `Destination Port Unreachable` от САМОГО роутера
**Наблюдение 2026-09-15 (подтверждено дважды в одной сессии).** С Mac (`192.168.6.104`) при пинге выключенного хоста верхнего сегмента:
```text
$ /sbin/ping -c 3 192.168.2.176
92 bytes from rasputin.lan (192.168.6.1): Destination Port Unreachable
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 5400 a0a3 0 0000 3f 01 509d 192.168.6.104 192.168.2.176
--- 192.168.2.176 ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet loss
```
**Как это читать (и как НЕ читать):**
| Признак | Значение |
|---|---|
| `Request timeout for icmp_seq N` | нормальный признак «хост выключен» |
| `Destination Port Unreachable` от `rasputin.lan (192.168.6.1)` | **это роутер отбивает попытку, а не ответ хоста** |
| Полный дамп IP-заголовка (`Vr HL TOS … Src/Dst`) | ответ сгенерирован ICMP-модулем роутера, а не удалённой машиной |
| `100.0% packet loss` | хост не ответил |
```text
/usr/sbin/arp -an | grep 176
? (192.168.6.176) at (incomplete) on en8 ifscope [ethernet]
```
`(incomplete)` = ARP-запрос ушёл, ответа нет → **L2-ответа нет вообще**, хост мёртв или вне сети. Это правильный признак для вывода «машина недоступна».
> ⚠️ **ПИТФОЛЛ ДИАГНОСТИКИ:** строка `Destination Port Unreachable` от роутера легко читается как «порт закрыт, машина жива». Это НЕВЕРНО. Она одинаково выглядит и для хоста, который выключен, и для хоста, который висит в kernel lockup без сетевого стека. Опираться нужно на **`arp (incomplete)` + 100% loss**, а не на текст ICMP-ошибки.
> ⚠️ Проверять достижимость узлов `192.168.2.0/24` из `192.168.6.0/24` надёжнее всего **тем же каналом, которым потом будете работать**: SSH по ключу. `ssh -o ConnectTimeout=10 -o BatchMode=yes root@192.168.2.176 'uptime'` — если отвечает, хост жив, и всё дальнейшее снимается через него. Пинг/веб-порты могут врать (см. `Reject to LAN` ниже), SSH — нет.
**Границы прав внутри сегментов (напоминание):**
- Router transit `192.168.6.0/24 ↔ 192.168.2.0/24` — разрешён правилами выше.
- Между сегментами есть санитарный `Reject to LAN` в `forward_lan` → произвольные порты не пройдут, ICMP-эхо тоже.
- Порты HA (`8123`) и SSH (`22`) через транзит роутера **не проверять как индикатор живости** — индикатор это сам факт ответа SSH-сессии.
## Downstream LAN Access
```text
firewall.@rule[11].name='Allow-downstream-LAN'
firewall.@rule[11].src='wan'
firewall.@rule[11].dest='lan'
firewall.@rule[11].target='ACCEPT'
firewall.@rule[11].src_ip='192.168.2.0/24' '192.168.0.1'
```
### KillSwitch
```text
firewall.@rule[12].name='KillSwitch'
firewall.@rule[12].src='lan'
firewall.@rule[12].dest='wan'
firewall.@rule[12].proto='all'
firewall.@rule[12].target='REJECT'
```
In nft `forward_lan`, ordering is important:
```text
1. Allow-Mac-cloudflared-QUIC-direct
2. Allow-upstream-LAN
3. KillSwitch
4. Accept lan to vpn forwarding
5. Accept DNAT port forwards
6. Reject to LAN
```
This ordering means only explicit `lan -> wan` exceptions escape; everything else either goes through VPN routing or is rejected from WAN.
Backup made before the Cloudflare firewall change:
```text
/etc/config/firewall.bak.20260714-195530
```
## Port Forwarding
There is a port-forward named `LocalSend-to-mbp-black-eth`:
```text
firewall.@redirect[0].name='LocalSend-to-mbp-black-eth'
firewall.@redirect[0].src='wan'
firewall.@redirect[0].dest='lan'
firewall.@redirect[0].src_ip='192.168.2.0/24'
firewall.@redirect[0].proto='tcp udp'
firewall.@redirect[0].src_dport='53317'
firewall.@redirect[0].dest_ip='192.168.6.104'
firewall.@redirect[0].dest_port='53317'
firewall.@redirect[0].target='DNAT'
```
Reflection rules exist for LAN clients hitting the router WAN IP on port `53317`, redirecting to `192.168.6.104:53317`.
## Cloudflare Tunnel / Mac Bypass Context
Problem addressed:
- Mac runs `cloudflared` for `zulip.qentra.top`.
- Router routes normal LAN traffic through VLESS via `tun0`.
- Cloudflare Tunnel connector traffic should go direct to WAN rather than through VLESS.
Chosen solution:
1. Mac `cloudflared` should use QUIC, not HTTP/2.
2. Cloudflare Tunnel QUIC is UDP destination port `7844`.
3. Router sends only Mac UDP/7844 through `main` table.
4. Firewall allows only Mac UDP/7844 from LAN to WAN before the KillSwitch.
Mac-side LaunchDaemon observed earlier:
```text
/Library/LaunchDaemons/com.cloudflare.cloudflared.plist
```
Original args observed before the change:
```text
cloudflared tunnel --protocol http2 run --token <redacted-cloudflare-tunnel-token>
```
Target Mac-side args:
```text
cloudflared tunnel --protocol quic run --token <redacted-cloudflare-tunnel-token>
```
Manual Mac commands provided for that change:
```sh
sudo cp /Library/LaunchDaemons/com.cloudflare.cloudflared.plist \
/Library/LaunchDaemons/com.cloudflare.cloudflared.plist.bak.$(date +%Y%m%d-%H%M%S)
sudo /usr/libexec/PlistBuddy \
-c "Set :ProgramArguments:3 quic" \
/Library/LaunchDaemons/com.cloudflare.cloudflared.plist
sudo launchctl bootout system /Library/LaunchDaemons/com.cloudflare.cloudflared.plist
sudo launchctl bootstrap system /Library/LaunchDaemons/com.cloudflare.cloudflared.plist
```
Verify on Mac:
```sh
plutil -p /Library/LaunchDaemons/com.cloudflare.cloudflared.plist
tail -50 /Library/Logs/com.cloudflare.cloudflared.err.log
```
Expected log clue:
```text
protocol=quic
```
Router-side counters can verify traffic is hitting the bypass:
```sh
nft list chain inet fw4 forward_lan
```
Look for increasing counters on:
```text
Allow-Mac-cloudflared-QUIC-direct
```
## Current Effective Traffic Behavior
```text
Mac 192.168.6.104 UDP/7844 -> main routing table -> WAN direct
Mac 192.168.6.173 UDP/7844 -> main routing table -> WAN direct
Any LAN client -> 192.168.2.0/24 -> main routing table -> WAN/upstream LAN
Other LAN traffic -> vpn routing table -> tun0 -> Xray/VLESS
LAN -> WAN not explicitly allowed -> rejected by KillSwitch
```
## Operational Commands
Restart VPN tunnel policy:
```sh
/etc/init.d/tunvpn restart
```
Restart firewall:
```sh
/etc/init.d/firewall restart
```
Show policy routing:
```sh
ip rule show
ip route show table vpn
ip route show table all
```
Show Cloudflare direct firewall counter:
```sh
nft list chain inet fw4 forward_lan
```
Show Xray logs:
```sh
tail -100 /tmp/xray-access.log
tail -100 /tmp/xray-error.log
```
Show active proxy processes:
```sh
ps w | grep -Ei 'xray|badvpn|tun2socks' | grep -v grep
```
Show current DHCP static leases:
```sh
uci show dhcp | grep -E 'mbp-black|192.168.6.104|192.168.6.173|00:e0:4c:68:04:88|66:02:b1:8b:2f:94'
```
## Recovery / Rollback
DHCP backup from Mac static lease change:
```text
/etc/config/dhcp.bak.20260714-195408
```
Routing service backup from Cloudflare direct-bypass change:
```text
/etc/init.d/tunvpn.bak.20260714-195530
```
Firewall backup from Cloudflare direct-bypass change:
```text
/etc/config/firewall.bak.20260714-195530
```
Rollback example:
```sh
cp /etc/init.d/tunvpn.bak.20260714-195530 /etc/init.d/tunvpn
chmod +x /etc/init.d/tunvpn
cp /etc/config/firewall.bak.20260714-195530 /etc/config/firewall
/etc/init.d/tunvpn restart
/etc/init.d/firewall restart
```
Rollback DHCP static leases:
```sh
cp /etc/config/dhcp.bak.20260714-195408 /etc/config/dhcp
uci commit dhcp
/etc/init.d/dnsmasq restart
```
## Notes And Pitfalls
- The Cloudflare bypass depends on Mac `cloudflared` using QUIC. If the Mac falls back to HTTP/2/TCP/443, the router exception will not match.
- There are two Mac source IPs because the Mac can use Ethernet or Wi-Fi.
- Both Mac IPs are now static DHCP reservations on Rasputin.
- `ip rule` can match `ipproto udp dport 7844` on this OpenWrt build; this was tested and confirmed before making the persistent change.
- The firewall allow alone would not be sufficient, because policy routing rule `101` sends LAN ingress to `table vpn`. The direct `ip rule`s at priorities `98` and `99` must remain before priority `101`.
- Rule order in `forward_lan` matters: Cloudflare direct allow must stay above `KillSwitch`.
- `eth0` was down at the 2026-07-22 capture; `usb0` was active and part of the `wan` firewall zone.
- 🔴 **Ping from Mac to an unreachable `192.168.2.x` host returns `Destination Port Unreachable` from `rasputin.lan (192.168.6.1)`, NOT a plain timeout.** This is the router rejecting the attempt — it does **not** mean the host is up with a closed port. Use `arp (incomplete)` + `100% packet loss` as the liveness signal. Confirmed 2026-09-15 (see the dedicated section above).
- Liveness of `192.168.2.0/24` hosts is best verified with the channel you intend to work through: key-based SSH (`ssh -o BatchMode=yes root@192.168.2.176 'uptime'`). Ping and web ports can mislead; an answered SSH session cannot.
- Secrets are redacted here intentionally. Consult the live router files only when the actual VLESS or Cloudflare credentials are needed.