Wrong source IP for udp packet generation #4971
Labels
Confirmed Bug
The bug reported is confirmed and able to be replicated.
Upstream
Something that needs to be resolved or addressed upstream.
Work in Progress
The issue is currently being worked on.
Describe the bug
GeyserMC not works properly on multiple IP servers.
I have two NIC on my PC
Routes:
Policy routing:
The bug:
GeyserMC reply with wrong source address
tcpdump logs from server
client sent packet to
192.168.2.100.19132
but server response with source ip192.168.1.100.19132
To Reproduce
Method 1
Same as above. Prepare two NIC on server, setup two routing table and policy routing on it.
Method 2
Setup multiple IP in same NIC
And the tcpdump logs:
This bug happens is because linux selects source address with following rules:
bind
orwrite
)destination IP
3-1. If
pref-src
is set on the route, use it3-2. if the route points a NIC, select an IP from it
3-3. If multiple IP on the NIC, use the ip marked as major.
In my first case, it hitted at rule 3-2. Because I have two routing table both covers client IP
61.231.185.43
, and it hit the first table. So that linux select an IP from NIC1 as the source address, but the packet is come from NIC2 and should hit another table.In my second case, it hits rule 3-3, multiple IP on same NIC. And
172.22.3.101
is marked assecondary
, so the kernel just selects172.22.3.100
TCP socket get covered by system at rule 2 but udp is not.
If the selection hits rule 3, it's basically guess. Because it lost the information of which IP is the client connect to.
It doesn't cause issue as client application cause server always send back whatever, but cause issue for a server application.
Expected behaviour
Similar to wireguard or etc udp based server, track udp session manually and reply with correct source address.
different from TCP session, system won't track udp session for user, we have to handle it by our self.
We have to bind specfic source IP on udp each session otherwise system just select the major IP from NICs based on default table. So that it brokes at system have multiple route table or secondary IP on the NIC.
In most case it won't be issue but I have multiple routing table on my system which causes problem.
It also cause problem for server have multiple IP address. This is very common on VPS or server which allowing us to purchase additional IP, the IP marked as secondary in the NIC are not useable at geyser.
Solution 1:
We should check
dst IP
for incoming packet, and bind it assrc IP
for the respons socket(each client needs one socket).wireguard-go
uses this solution. It saves thedst ip
for incoming packets for each peers with custom designedsticky socket
.Solution 2:
Allowing us config multiple bind address in the config and spawn multiple udp socket for each address instead of bind
0.0.0.0
with one socket.So that each socket only listens packets from corresponding NIC and we can send response packet with corresponding socket to make sure the
src IP
is correct.bind9
uses this solution. It doesn’t listen 0.0.0.0, but searches all IP on all NIC and create a socket for each of them.Screenshots / Videos
No response
Server Version and Plugins
No response
Geyser Dump
https://dump.geysermc.org/kCwMeUAIXZxu4OtZHMazGI2WF60M8hiI
Geyser Version
5.2.3-SNAPSHOT (b109-49bd564)
Minecraft: Bedrock Edition Device/Version
No response
Additional Context
No response
The text was updated successfully, but these errors were encountered: