Hi @pyrainath ,
That likely explains the slowness.
When the TCP buffer is full it can’t accept more data. The sender has to pause or slow down, even though it could send more.
Your server or the client is likely advertising a small TCP receive window and/or not scaling it efficiently. This could explain the slowness over the WAN link.
Enabling/Fixing the window scaling or buffer sizes could improve throughput speed.
Run PCAP on both the server and a client to compare advertised window sizes and verify where/if you can improve it.
The standard TCP window size without scaling is limited to ~64 KB. That works fine on a LAN with low latency, but is way too small for high-speed WAN links with higher latency. Enabling TCP Window Scaling (RFC 1323) allows the receiver to advertise a much larger window (up to 1 GB).
On your Local LAN you have low latency which means the small TCP window fills and empties quickly. So even if the window is small, there's enough round-trips per second to maintain good throughput which is why you could see speeds like 500 Mbps inside your network.
However, WAN links usually have much higher latency and when the TCP window is full, the sender has to wait longer for an ACK before sending more data. To fix this you need a larger window to keep the pipe full over longer distances.
Hope this helps,
-Kim.
... View more
