Why is SMB traffic slow ?

[kiwi]

Every once in a while, there's a returning question on why SMB traffic is so slow. In this blog, I'll highlight a couple of solutions. Proposed by both community members and TAC engineers, several community members have found these useful and they've helped solve issues in the past.

Searching the LIVE community will already provide you with a couple of links featuring some related discussions and KB articles. These might already provide you a solution or give you some guidance to troubleshoot if you're experiencing this issue:

• 97% Speed Decrease On SMB Traffic PAN-OS 8.1 (discussion)
• How to Improve Performance for Protocols like SMB and FTP Without Application Override (KB article)
• GlobalProtect SMB Traffic Slowness (discussion)
• GlobalProtect SSL VPN Slow SMB Transfers  (discussion)

Allow me to first explain why SMB is a bit of a special protocol and why it's behaving the way it is:

SMB content is inspected differently compared to other protocols, like HTTP or FTP for example. The SMB decoder is unable to implement suspend since file transfers are done in a block-based manner, requiring continuous CTD inspection to follow the protocol on each block. Suspending only for one file could allow evasion for all subsequent files in the same session. ForSMB, every payload is scanned for content inspection and there is no offload mechanism to increase speed. This generally leads to a decreased throughput.

Reading the above already hints to a possible solution/workaround. You can disable content inspection by adding an app-override for this specific traffic, this will allow the session through using fast-path. This approach should be used only if other fail safes are in place, and only between trusted hosts:

Policies > Application Override 

Tips & Tricks: How to Create an Application Override 

In case an App Override is not possible because L-7 inspection is required, an alternative workaround would be to disable server response inspection (DSRI).  SMB traffic is very chatty by nature.  By default, traffic in both directions from client-to-server (C2S) and from server-to-client (S2C) will be inspected.  With DSRI, the firewall will only inspect C2S traffic !

Policies > Security > Actions

 Typically, DSRI is used in environments where internal servers are trusted !!

Note that DSRI is not limited to SMB traffic and can be used on other scenarios as well:

DotW: Using DSRI with the Palo Alto Networks firewall

Improving Performance of HTTP with DSRI 

Make sure you're on a recommended PAN-OS release to ensure any issues encountered are not caused by software. So check out the PAN-OS Software Release Guidance to confirm you're running the recommended PAN-OS release or if you are in need of an upgrade to rule out eventual bugs from previous software versions.

Sometimes you might have to dig somewhat deeper or even have Palo Alto Networks Support look into the issue.  Here are some additional debugging steps or troubleshooting tips you can look into:

• If you have the option, try different protocols (SCP, FTP, HTTP) to make sure that it's an SMB decoder issue.
• If you're testing with SMBv3, try disabling multichannel on the Windows server and client. Because of the way that SMBv3 multi-channel works in splitting up files Palo Alto Networks recommends disabling SMB multi-channel through the Windows PowerShell. For more information on this task, please refer to: technet.microsoft.com/en-us/library/dn610980(v=ws.11).aspx
• "show counter global filter delta yes" is your friend! While the file transfer is ongoing, run this command a couple of times and check if you see suspicious counter building up.
• Check if the packet descriptors (on-chip) are spiking during the file transfer. You want make sure that SMB sessions aren't using a high percentage of pkt buffers during the file transfer. Use the CLI command show running resource-monitor to verify this.
• Things to collect for analysis: "show session id (session id)" of a few of those sessions + generate tech support file.

Can you think of additional debugging/troubleshooting steps or workarounds for this popular topic? Please share your experience and possible solutions in the comments section below!

Thanks for taking time to read this blog.
If you enjoyed this, please hit the Like (thumb up) button, don't forget to subscribe to the LIVEcommunity Blog area.

Kiwi out!