Slow speed with GlobalProtect

[nanukanu]

Hi to all,

We are trying to understand why the download speed is really slow vía GP.

We stablish a VPN GP with IPsec without Split Tunneling. We acces to some public web to download a test file. Im downloading a 1G file.

If we download without GP but through the Palo Alto we achieve 60MB/s, but vía GP we achieve maybe 6 or 10MB/s.

Any idea about that? Someona has experienced this issue? The differents shows issued don't present mss or mtu problems.

Thank you!

[DigitalAffinity]

Which model of firewall are you using?

Do you have threat profiles enabled for that traffic?

Each firewall model has a throughput limit based on the horsepower of the firewall. If you run traffic through the threat stack, it slows it down more. For example, I had a PA-500 and the max throughput is rated at 100mbps for traffic running through the threat stack. That equates to about 10/MB a second. Also, you have overhead from the IPSec encryption, that will slow down most traffic as well.

It is really the price you pay for security. Running traffic through the VPN and firewall means that additional overhead gets added on the traffic. Here are some ideas if you want:

- enable some routes in GP to exclude traffic to certain sites from the VPN

- If you have a GP portal license, you can turn on "exclude video" sites and it won't slow down video.

- create policies to disable the threat stack for certain type of traffic

[nanukanu]

The PA850 sheet talks about 500Mbps (62MB/s) IPSec throughput. There are NO profiles attached to the rule, the throughput following the datasheet without profiles are about 2Gbps...

If you hace 100mbps as you said this is 12MB/s so if you are receiving 10MBs this is acceptable.

So, i don't understand where the bottleneck happens. It's nothing related about price fot security.. it's just specs.

Thank you,

[DigitalAffinity]

I understand. The specs for your firewall are high enough. I was just relating my experience where traffic outside of the GP VPN was fast, but traffic over the VPN is slow. That seems to be a universal condition introduced by the overhead of the VPN encryption. That is what I meant by price of security; VPN traffic won't be as fast as regular firewall traffic in my experience. Maybe there is some setting that makes it faster, but I'd love to know that as well. So far across two different PA firewalls, VPN traffic is slower.

[nanukanu]

Let's if someone can add more light to this subject. I know about the overhead, for this reason I check-in MTU or MSS but the performance it's not acceptable. Datasheet says 500mbps... The are a lot of difference between 62MBs expected vs 5-6MBs...

Thank you,

[Brandon_Wertz]

So as my recent efforts have lead me down this path, this is a very difficult question to answer.

When I first deployed GP using SSL only type tunnel on my 5220 (With decryption) I noticed a significant disparity in my speedtest results.  At home without GP I would get between 90-105Mbps.  With GP SSL type VPN tunnel I would get between 10-25Mbps.  I opened a few tickets with TAC and got various answers and "fixes."

The best fix I've since deployed is converting the tunnel type to allow IPSec VPN for clients. This setting got me to around 50-65Mbps on a speedtest.

In one of my tickets with TAC they modified the tunnel I use for VPN to a lower amount to account for the encryption overhead (something I hadn't done.)  There were varied settings here going down to 1424 down to as low as 1350 which is where I'm currently at.  This was done to help prevent fragmentation of the "transfer stream."  When this particular change was made I didn't really notice a significant change in throughput capacity.

In one of my tickets an engineer made a great observation, speed test sites aren't really a good measure of "throughput" or capacity while on VPN.  They have their own unique setup that doesn't necessarily jive well for a VPN client.

Your best bet is to use a site that tests as if you're transferring a file.  So in my case I feel getting 50-65Mbps via VPN w/o split-tunneling is more than enough.  

Here is a better site to use:  https://www.thinkbroadband.com/download 

[Brandon_Wertz]

If you're trying to do this on a PA-850 and expecting high throughput I wouldn't.  There are A LOT of factors that go into a devices performance specs.  

I'm assuming your box is doing more than just supporting this single user GP connection.  If your client is getting 48Mbps (6MB) of total potential capacity of 500Mbps, what other traffic is the box doing?

[nanukanu]

Hi,

Thank you for you reply.

This device is in test environment prior to full migration. So unfortunatelly yes, no other traffic is crossing the firewall.

I don't trust on those speedtest, the test im doing it's director download vía browser from some website that offers Free test downloads.
Where you tune the tunnel capacity?

Regards,

[gwesson]

The testing you are doing is flawed, at least when compared with a full traffic test. The reason is based on the number of sessions and how they are handled within the dataplane of the firewall.

If you're using a single download from your browser, you are only using a single TCP connection to actually get the content. You'll have multiple connections when browsing the site, but that one object being downloaded is limited by many factors.

If you had 50 PCs downloading 50 different objects from different servers, then you'll have a much more accurate test. Sessions can be distributed across dataplanes and cores, for example.

[nanukanu]

Hi,
Thank you for your response. Im try to explain better the scenario.

1st Test. One computer behind the firewall (Site A) downloading a file from https://www.thinkbroadband.com/download I achieve acceptable speed. Im the only one using the firewall, no security profiles attached to the rule.

2n Test. From a computer in a remote site (Site B) and download the same file, I achieve better results due to the best line in Site B.

3rd Test From Site B establish a GP IPsec ti Site A without Split tunneling, so all traffic crosses the FW. I go to the same Site and download the same file, the download speed dramatically slowdown.

In 2nd and 3rd scenario I'm the only crossing the FW.

Regards,

[nanukanu]

Hi,
Thank you for your response. Im try to explain better the scenario.

1st Test. One computer behind the firewall (Site A) downloading a file from https://www.thinkbroadband.com/download I achieve acceptable speed. Im the only one using the firewall, no security profiles attached to the rule.

2n Test. From a computer in a remote site (Site B) and download the same file, I achieve better results due to the best line in Site B.

3rd Test From Site B establish a GP IPsec to Site A without Split tunneling, so all traffic crosses the FW. I go to the same Site and download the same file, the download speed dramatically slowdown.

In 2nd and 3rd scenario I'm the only one crossing the FW.

Regards,

[gwesson]

In all three scenarios, you're still using a single download. There are bottlenecks that can be associated with a single session (and even more when it comes to IPSec). Things like encapsulation/decapsulation and how it is split across multiple cores, and a whole host of things.

The datasheet numbers are for scale, not for individual downloads from a single client. 

I don't want to come across as harsh, I just can't think of a better way to say it. A single download is VASTLY different than real clients connecting and doing regular work.

If you insist on using this type of flawed testing, at the very least try doing several downloads at the same time from different sites on that test client at site B. At least then you'll be using multiple sessions to fill the tunnel better.

[nanukanu]

Hi Gwesson,

Thank you for your words,

I understand how It works, thank you for your explanation.
But the fact is that the test that I am doing is the way of working that our customer has.
Workers in remote sites download files via HTTP and via FTP through Global Protect and L2L VPN and it is difficult to download the same file in múltiple parts as you are suggesting.

I am just surprised by the degradation of speed in a device with these characteristics, when the current equipment, much more older (ASA FW) and through VPN supports speeds by far much better doing the same.

Appreciate for your help,

Thank you,

[Adam42]

Hello @nanukanu I hope you did find a solution for your issue 

I'm facing the same issue using PA-200  but the Upload is quite good  but the download traffic is slowing down (from 30 Mbps to 2Mbps).

SpeedTest Without Firewall 30 Mbps

SpeedTest with Firewall PA-200 2Mbps 

[HenriqueGurgel]

Hi @Adam42 ,

I am facing a very similar issue as you are.

Please could you tell which version of the firewall and the global protect you are using. I am suspecting of some version bug and I would like to know if you the version you are using is the same I am.

Thanks