AXE-16000 jumbo frame does not work properly on 10G interface

[AlanHell]

Hi,
I have a Debian 11 (Linux) PC connected directed to the 10G port of the router. The other 10G port on the router is connected to my local 10G switch which has my NAS and other PCs connected. All 10G interface from my NAS and PCs has their MTU set to 9000, including the switch itself.

Since the AUSU router does not allow me to configure MTU size, I just enable jumbo frame under the LAN configuration and expected it to work. I have run some iperf speed test between my windows PC(connected to the 10G switch) and the Linux machine (connected to the ASUS router) , and is able to reach 10G line rate. Hence I think the network itself works just fine. However, when I try to mount my NAS drives (cifs/samba V3.0) on the Linux machine, the connection will time out.

The reason I point out MTU, is because when I set the MTU size on the Linux machine back to 1500, I will be able to mount the drive. When I keep the MTU to 9000, but move the cable from the ASUS router to the local 10G switch, it also works. It seems that whenever I pass the network drive traffic through the 2 10G port on the ASUS router using frame size close to 9000, it starts to give me trouble.

Did any one know what is the actual max frame size this "jumbo frame" feature supported? Is it a bug or just a feature limitation? I really would like to set my MTU size to 9000 as it is much faster than the default 1500, and it is kind of a waste to use small frame size for 10G interface anyway.

[jzchen]

Interesting setup. I would personally attach the NAS to one of the ASUS router 10 G ports, the 10 G switch to the other router 10 G port, and connect the computers to the switch. Did you do it this way to test the MTU capabilities of the AXE16000?

Some people have noted the 10 G ports on the router are not capable of 10 G speed, so I find comfort that you were able to see this throughput. I personally would not be concerned about MTU unless it was slow/not 10 G speed...

[AlanHell]

jzchen wrote:
Interesting setup. I would personally attach the NAS to one of the ASUS router 10 G ports, the 10 G switch to the other router 10 G port, and connect the computers to the switch. Did you do it this way to test the MTU capabilities of the AXE16000?

Some people have noted the 10 G ports on the router are not capable of 10 G speed, so I find comfort that you were able to see this throughput. I personally would not be concerned about MTU unless it was slow/not 10 G speed...

I use Iperf to test the speed, and by setting MTU to 9000, I am able to get very close to 10G speed. If I just use default 1500, it is around 7.8G to 8.5G. I do not thing using NAS or Linux PC matter in this case as both of them are using the same 10G PCIe NIC, so if it does not work, then it does not. As the wifi is not going to reach 10G anyway, I'd rather put most of my 10G stuff on the switch as it has all the advance layer 2 and layer 3 features, not like the ASUS.

[Saltgrass]

I was wondering what ISP speed you have? If it were to be 2.5 Gbs or higher, maybe using the 10 Gbs port for your Wan port would have some effect.

[jzchen]

How about with the computers both connected to the switch and MTU set to 9000 on both, are they able to connect to the internet/www?

I did a little reading on MTU/jumbo packets and the limitation may be set by your ISP. Â*Since the router is forwarding packets from the ISP if the ISP sets MTU to 1500 that will be the limit for ports connected to it. Â*Since the switch is independent it can support the 9000 among devices connected directly to it. Â*Just wondering then if the internet is accessible as you’ve forced your computers to 9000 but the ISP is probably set to 1500…

[AlanHell]

jzchen wrote:
How about with the computers both connected to the switch and MTU set to 9000 on both, are they able to connect to the internet/www?

I did a little reading on MTU/jumbo packets and the limitation may be set by your ISP. Â*Since the router is forwarding packets from the ISP if the ISP sets MTU to 1500 that will be the limit for ports connected to it. Â*Since the switch is independent it can support the 9000 among devices connected directly to it. Â*Just wondering then if the internet is accessible as you’ve forced your computers to 9000 but the ISP is probably set to 1500…

It is actually the other way around. When running 9000 MTU through the 10G interface, I am able to access internet just fine. It is only when I try to mount the NAS drive connected to a local switch, it becomes very slow. I think the 9000 MTU is only enforced by my local 10G switch, which clearly is not working with the ASUS router. I assume when I sent internet request directly to the modem, it just automatically fragment it to 1500 either from the router or the modem.

[AlanHell]

Saltgrass wrote:
I was wondering what ISP speed you have? If it were to be 2.5 Gbs or higher, maybe using the 10 Gbs port for your Wan port would have some effect.

My ISP is slow, it is only 1G down, and 50M up. So my 2.5G WAN port is set to 1G since that is what my modem has. The 10G is only used for my internal network, as I have a quite fast NAS, and is running Plex for 4K streaming. The problem is not that it does not work at all. It is that as soon as I enabled jumb frame, all my net drive becomes very slow to a point that is not usable. And as soon as I disable jumbo frame, or set my NIC MTU to 1500, the problem went away. To me, this seems to be a jumbo frame issue. However, there is no documentation on this feature from ASUS, so I am not sure if it is fixable by firmware update, or just hardware limitation.