ip分流 on 華鳥風月 https://blog.9-ch.com/tags/ip%E5%88%86%E6%B5%81/ Recent content in ip分流 on 華鳥風月 Hugo -- gohugo.io zh-cn Thu, 09 Mar 2023 15:31:37 +0800 RouterOS 通过 OSPF IP分流非CN流量 https://blog.9-ch.com/post/homelab-no-cn-routes/ Thu, 09 Mar 2023 15:31:37 +0800 https://blog.9-ch.com/post/homelab-no-cn-routes/ ROS ip 分流方案 主要适用于不想折腾PS/XBOX/Switch设备网关配置,又不希望出口网关为卵漏油/AIB的,或者怕折腾网络时家里断网影响家人的场景

更新

  • 2022-06-18: 高版本ROS建立邻居困难问题建议尝试PTP/PTMP模式,并且对齐两侧参数(hello、dead count等),并检查224.0.0.0组播地址范围有无防火墙/路由规则影响导致的不可达问题
  • 2023-01-28: 更新探针脚本, 调整文档结构

相关硬件

  • Mikrotik AC^2 (ros stable 7.2.3*)

  • NUC5i5MYBE (debian 11)

    *该版本OSPF似乎有bug,测试邻居离线时出现过一次不能自动删除路由,建议还是LT版本

网络结构

1
2
3
4
5
6
7

AC^2--ETH1(192.168.0.254)--(192.168.0.1)光猫--Internet
    |
    |-ETH2-| birdge:lan
    |-ETH3-| ip:192.168.1.1
    |-ETH4-|----> 192.168.1.0/24 SW
    |
    |-ETH5(192.168.255.1)--(192.168.255.254)NUC

相关网段划分:

  • 192.168.0.0/24 –光猫/WAN口网段
  • 192.168.1.0/24 –普通设备网段
  • 192.168.255.0/24 –NUC网段

基础防火墙配置

不建议无脑抄,大部分配置有comment,建议按需添加

LAN包含lan桥ether5,WAN包括pppoe-out

ROS IPV4防火墙(较长已折叠) 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

/ip firewall address-list
add address=0.0.0.0/8 comment="defconf: RFC6890" list=no_forward_ipv4
add address=169.254.0.0/16 comment="defconf: RFC6890" list=no_forward_ipv4
add address=224.0.0.0/4 comment="defconf: multicast" list=no_forward_ipv4
add address=255.255.255.255 comment="defconf: RFC6890" list=no_forward_ipv4
add address=192.168.233.233 list=BOT
add address=192.168.0.0/16 list=private
add address=10.0.0.0/8 list=private
add address=172.16.0.0/12 list=private
/ip firewall filter
add action=drop chain=input comment=ANTI-BOT disabled=yes src-address-list=\
    BOT
add action=accept chain=input comment=Accept-OSPF protocol=ospf
add action=accept chain=input comment=Accept-LAN-DNS dst-address=192.168.1.1 \
    dst-port=53 protocol=udp
add action=accept chain=input comment=Accept-NTP dst-port=123 protocol=udp
add action=drop chain=input comment=BLOCK-DNS dst-port=53 in-interface-list=\
    WAN protocol=udp
add action=drop chain=input comment=BLOCK-DoT dst-port=53 in-interface-list=\
    WAN protocol=tcp
add action=drop chain=input comment=BLOCK-HTTP dst-port=80 in-interface-list=\
    WAN protocol=tcp
add action=drop chain=input comment=BLOCK-HTTPS dst-port=443 \
    in-interface-list=WAN protocol=tcp
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMP" log-prefix=\
    IPv4-ping protocol=icmp
add action=accept chain=input comment=\
    "defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
    ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN
add action=drop chain=forward comment="defconf: drop bad forward IPs" \
    src-address-list=no_forward_ipv4
add action=drop chain=forward comment="defconf: drop bad forward IPs" \
    dst-address-list=no_forward_ipv4
/ip firewall nat
add action=masquerade chain=srcnat comment="Hairpin NAT" dst-address=\
    192.168.1.0/24 out-interface=lan src-address=192.168.1.0/24
add action=masquerade chain=srcnat comment=ONU dst-address=192.168.0.0/24 \
    src-address=192.168.1.0/24
add action=masquerade chain=srcnat comment="defconf: masquerade" \
    ipsec-policy=out,none out-interface-list=WAN

DNS解析

ROS配置

使用netwatch检查NUC状态,离线时自动将dns切换为公共dns

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

/system script
add dont-require-permissions=yes name=use-default-dns owner=admin policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon source="/\
    ip/dns/set allow-remote-requests=yes cache-size=4096KiB servers=119.29.29.\
    29,223.5.5.5 \r\
    \n/ip dns cache flush"
add dont-require-permissions=yes name=use-sgw-dns owner=admin policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon source="/\
    ip/dns/set allow-remote-requests=yes cache-size=4096KiB servers=192.168.25\
    5.254\r\
    \n/ip dns cache flush"
/tool netwatch
add down-script=use-default-dns host=192.168.255.254 interval=30s up-script=use-sgw-dns

(可选)192.168.1.0/24网段dns劫持

1
2
3
4

/ip firewall nat
add action=dst-nat chain=dstnat comment="Hijacking for DNS" dst-port=53 \
    protocol=udp src-address=192.168.1.0/24 to-addresses=192.168.1.1 \
    to-ports=53

Debian配置

SmartDNS分流解析

1

apt install smartdns

配置/etc/smartdns/smartdns.conf 如下

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

bind :53
cache-size 0
prefetch-domain no
serve-expired no
speed-check-mode ping,tcp:80

log-level info
log-file /var/log/smartdns.log
server 223.5.5.5:53
server 119.29.29.29:53
server-https https://dns.google/dns-query -check-edns -blacklist-ip -group GFW -exclude-default-group
server-https https://1.1.1.1/dns-query -blacklist-ip -group GFW -exclude-default-group
server-https https://cloudflare-dns.com/dns-query -blacklist-ip -group GFW -exclude-default-group
conf-file /etc/smartdns/gfw.conf

/etc/smartdns/gfw.conf 使用crontab更新

1

0 2 * * * curl -s https://raw.githubusercontents.com/RyoLee/gfwlist2smartdns/master/run.sh | bash -s -- -o /etc/smartdns/gfw.conf && /etc/init.d/smartdns reload

最终效果list内走223.5.5.5/119.29.29.29,list外走DoH

OSPF路由

ROS配置

添加防火墙放行

1
2
3
4
5

/ip firewall mangle
add action=mark-routing chain=prerouting new-routing-mark=main passthrough=\
    yes protocol=ospf
add action=mark-routing chain=prerouting dst-address-list=!private \
    in-interface=ether5 new-routing-mark=bypass passthrough=yes

Bypass路由表 & OSPF

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

/routing ospf instance
add disabled=no name=sgw router-id=192.168.255.1 routing-table=main
/routing ospf area
add disabled=no instance=sgw name=sgw
/routing table
add disabled=no fib name=bypass
/ip route
add comment="BYPASS" disabled=no distance=1 dst-address=0.0.0.0/0 \
    gateway=pppoe-out pref-src=0.0.0.0 routing-table=bypass scope=30 \
    suppress-hw-offload=no target-scope=10
/routing ospf interface-template
add area=sgw disabled=no interfaces=ether5
/routing rule
add action=lookup disabled=no routing-mark=bypass table=bypass
add action=lookup disabled=no table=main

Debian配置

安装bird2

1

apt install bird2

配置/etc/bird/bird.conf (网卡为enp0s25)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

log syslog all;

router id 192.168.255.254;


protocol device {
	scan time 60;
}

protocol kernel {
	ipv4 {
	      import none;
	      export all;
	};
}

protocol static {
	ipv4;
	include "routes4.conf";
}

protocol ospf v2 {
    ecmp no;
  	ipv4 {
		export all;
	};
	area 0.0.0.0 {
		interface "enp0s25" {
			type broadcast;
		};
	};
}

非CN流量出口为虚拟设备为utun,若使用TProxy则配置为对应的以太网网卡(如本例中enp0s25)

/etc/bird/routes4.conf 使用crontab更新

1
2

0 2 * * * curl -s https://api.9-ch.com/ncr?device=utun  -o /etc/bird/routes4.conf
0 3 */1 * * /etc/init.d/bird reload

心跳检查,若google 204服务检查失败则停止bird触发回落

探针脚本路径 /opt/check/tun.sh (已删除bark推送部分, 可自行按需添加)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

#!/usr/bin/bash
COUNT=0
MAX_COUNT=6
while [ $COUNT -lt $MAX_COUNT ]
do
    SER=0
    NET=0
    if [ $(curl --connect-timeout 10 --interface utun -w "%{http_code}" -s https://www.google.com/generate_204) -eq 204 ];then
        NET=1
    fi
    if /etc/init.d/bird status|grep Active|grep -q running;then
        SER=1
    fi
    if [ $NET -eq 1 ] && [ $SER -eq 0 ];then
        /etc/init.d/bird start
        # curl -X "POST" "https://******/push" -H 'Content-Type: application/json; charset=utf-8' -d $'{"body": "***隧道已上线","device_key": "******","level": "passive","title": "网络探针"}'
        exit 0
    fi
    if [ $NET -eq 0 ] && [ $SER -eq 1 ];then
        let COUNT+=1
        if [ $COUNT -eq $MAX_COUNT ];then
            /etc/init.d/bird stop
            # curl -X "POST" "https://******/push" -H 'Content-Type: application/json; charset=utf-8' -d $'{"body": "***隧道已离线","device_key": "******","level": "passive","title": "网络探针"}'
            exit 0
        fi
        continue
    fi
    exit 0
done

1

* * * * * bash /opt/check/tun.sh

ROS DDNS失效问题

见前文ros-ddns失效

参考信息

同网段分流openwrt方案

见前文RouterOS 通过 OSPF IP分流至 OpenWrt

零散问题比较多, 本方案为基于以上测试经验改进出新方案,非CN IP流量出口改为使用运行debian的一台NUC

]]> RouterOS 通过 OSPF IP分流至 OpenWrt https://blog.9-ch.com/post/homelab-ros-ospf/ Wed, 20 Apr 2022 17:59:11 +0800 https://blog.9-ch.com/post/homelab-ros-ospf/ ROS ip 分流方案 更新

组网参考

Reflist:

主要适用于不想折腾PS/XBOX/Switch设备网关配置,又不希望出口网关为卵漏油/AIB的,或者怕折腾网络时家里断网影响家人的

遇到一些问题记录一下

ROS DDNS失效

表现: 域名解析ip不正确

原因: 分流后ROS内置ip cloud(DDNS)流量同样会被指向OpenWrt导致错误,尝试打标签bypass对应udp流量未成功,因为有公网IP所以直接改为通过域名服务商API更新对应解析为PPPOE接口地址

解决: 修改以下脚本中"CloudFlare variables"段内容即可.原始脚本见Mikrotik_CF_DDNS

Cloudflare DDNS ROS脚本(IPV4) 
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120

#########################################################################
#         ==================================================            #
#         $ Mikrotik RouterOS update script for CloudFlare $            #
#         ==================================================            #
#                                                                       #
# - You need a CloudFlare account & api key (look under settings),      #
#   a zone and A record in it                                           #
# - All variables in first section are obvious,                         #
#   except CFid and CFzoneid                                            #
# - Obtain CFzoneid from Cloudflare Dashboard,                          #
#   on Overview tab scroll down                                         # 
#   To obtain CFid use following command in any unix shell:             #
#    curl -X GET "https://api.cloudflare.com/client/v4/zones/YOUR_ZONE_ID/dns_records?name=YOUR_DOMAIN" -H "Authorization:Bearer $CFtkn" -H "Content-Type: application/json" | python -mjson.tool
# - You can use my Postman script to get those variables                #
# - Enable CFDebug if needed - it'll print some info to logs            #
# - Enable CFcloud if you don't get a public IP on interface            #
# - Put script under /system scripts giving "read,write,ftp" policy access.       #
#   For 6.29 and older "test" policy is also needed.                    #
# - Add script to /system scheduler using it's name in "on-event"       #
# - Requires at least RouterOS 6.44beta75 for multiple header support   #
#                                                                       #
#              Credits for Samuel Tegenfeldt, CC BY-SA 3.0              #
#                        Modified by kiler129                           #
#                        Modified by viritt                             #
#                        Modified by asuna                              #
#                        Modified by mike6715b                          #
#                                                                       #
#               Tested and working as of February 22, 2021              #
#########################################################################

################# CloudFlare variables #################
:local CFDebug "false"
:local CFcloud "false"

:global WANInterface "[YOUR_WAN_INTERFACE_NAME]"

:local CFdomain "[YOUR_DOMAIN]"

:local CFtkn "[YOUR_CF_TOKEN]"

:local CFzoneid "[YOUR_CF_ZONE_ID]"
:local CFid "[YOUR_CF_ID]"

:local CFrecordType ""
:set CFrecordType "A"

:local CFrecordTTL ""
:set CFrecordTTL "60"

#########################################################################
########################  DO NOT EDIT BELOW  ############################
#########################################################################

:log info "Updating $CFDomain ..."

################# Internal variables #################
:local previousIP ""
:global WANip ""

################# Build CF API Url (v4) #################
:local CFurl "https://api.cloudflare.com/client/v4/zones/"
:set CFurl ($CFurl . "$CFzoneid/dns_records/$CFid");
 
################# Get or set previous IP-variables #################
:if ($CFcloud = "true") do={
    :set WANip [/ip cloud get public-address]
};

:if ($CFcloud = "false") do={
    :local currentIP [/ip address get [/ip address find interface=$WANInterface ] address];
    :set WANip [:pick $currentIP 0 [:find $currentIP "/"]];
};

:if ([/file find name=ddns.tmp.txt] = "") do={
    :log error "No previous ip address file found, createing..."
    :set previousIP $WANip;
    :execute script=":put $WANip" file="ddns.tmp";
    :log info ("CF: Updating CF, setting $CFDomain = $WANip")
    /tool fetch http-method=put mode=https output=none url="$CFurl" http-header-field="Authorization:Bearer $CFtkn,content-type:application/json" http-data="{\"type\":\"$CFrecordType\",\"name\":\"$CFdomain\",\"ttl\":$CFrecordTTL,\"content\":\"$WANip\"}"
    :error message="No previous ip address file found."
} else={
    :if ( [/file get [/file find name=ddns.tmp.txt] size] > 0 ) do={ 
    :global content [/file get [/file find name="ddns.tmp.txt"] contents] ;
    :global contentLen [ :len $content ] ;  
    :global lineEnd 0;
    :global line "";
    :global lastEnd 0;   
            :set lineEnd [:find $content "\n" $lastEnd ] ;
            :set line [:pick $content $lastEnd $lineEnd] ;
            :set lastEnd ( $lineEnd + 1 ) ;   
            :if ( [:pick $line 0 1] != "#" ) do={   
                #:local previousIP [:pick $line 0 $lineEnd ]
                :set previousIP [:pick $line 0 $lineEnd ];
                :set previousIP [:pick $previousIP 0 [:find $previousIP "\r"]];
            }
    }
}

######## Write debug info to log #################
:if ($CFDebug = "true") do={
 :log info ("CF: hostname = $CFdomain")
 :log info ("CF: previousIP = $previousIP")
 :log info ("CF: currentIP = $currentIP")
 :log info ("CF: WANip = $WANip")
 :log info ("CF: CFurl = $CFurl&content=$WANip")
 :log info ("CF: Command = \"/tool fetch http-method=put mode=https url=\"$CFurl\" http-header-field="Authorization:Bearer $CFtkn,content-type:application/json" output=none http-data=\"{\"type\":\"$CFrecordType\",\"name\":\"$CFdomain\",\"ttl\":$CFrecordTTL,\"content\":\"$WANip\"}\"")
};
  
######## Compare and update CF if necessary #####
:if ($previousIP != $WANip) do={
 :log info ("CF: Updating CF, setting $CFDomain = $WANip")
 /tool fetch http-method=put mode=https url="$CFurl" http-header-field="Authorization:Bearer $CFtkn,content-type:application/json" output=none http-data="{\"type\":\"$CFrecordType\",\"name\":\"$CFdomain\",\"ttl\":$CFrecordTTL,\"content\":\"$WANip\"}"
 /ip dns cache flush
    :if ( [/file get [/file find name=ddns.tmp.txt] size] > 0 ) do={
        /file remove ddns.tmp.txt
        :execute script=":put $WANip" file="ddns.tmp"
    }
} else={
 :log info "CF: No Update Needed!"
}

TCP流量握手缓慢

表现: 访问网站时打开缓慢,F12查看SSL阶段时间长度接近甚至超过10秒

原因: OP与ROS在同一网段,进出流量路由不对称,触发ROS防火墙动作

解决: 禁用ROS防火墙drop invalid规则(位于ip-firewall-filter rules)

ICMP(Ping)异常

表现: ping 被分流网段ip时黑洞

原因: 出口Op通过TProxy只能处理TCP/UDP流量

解决:

  • A: Op添加 ICMP劫持(假响应,可规避一些软件的报错)
  • B: ROS给ICMP流量打上bypass的route标签,再创建一个bypass路由表,该表不包含OSPF宣告的分流规则,所以如果没有被ISP之类的劫持就是裸连的真响应

Openwrt无法与被分流网段通信(循环)

表现: OpenWrt与被分流网段通信时不通

原因: ROS接到宣告的表后循环Op->ROS->Op->…

解决: 给Op流量打上标签,直接走bypass表

防火墙规则(Mangle)可参考

1

chain=prerouting action=mark-routing new-routing-mark=bypass passthrough=yes src-address=[BYPASS_DEVICE_IP] dst-address-list=!private log=no log-prefix=""

DNS CDN优化

表现: 不需要分流区域域名CDN解析为分流区域

原因: Op全局流量隧道,DNS解析也被隧道处理,导致CDN失效

解决: dnsmasq-full分流list外走国内DNS服务,list内走DoH,Op故障时回落全国内DNS服务

OpenWrt访问内网异常

表现: Op所在子网广播域外的其他子网访问Op时,无法正常响应

原因: Op走了bypass表,未添加其他子网路由,下一跳直接从pppoe-out出去了

解决: 防火墙匹配src address为op的ip,dst address为需要支持访问子网流量打上main标签走main路由表

]]>