Content update 709 revoked?

[Anon1]

All firewalls automatically downgraded content version from 709 to 708. Was 709 revoked? Anybody else having the same behavior?

[SuryaR · Accepted solution]

Yes. The update was revoked.

More details here:

https://live.paloaltonetworks.com/t5/Customer-Advisories/Important-information-regarding-Content-App...

[Anon1]

Thank you for the link. I wish PA would send information email to customers like for new content released.

[Brandon_Wertz · Accepted solution]

@Anon1, they do.  Do you not get e-mails about new content signatures being released?

[Anon1]

Hello Brandon,

yes, I get email notifications when new content updates are released.

I also got an email "Important customer update regarding content version 709" with a link to the mentioned article in the meanwhile. However, I would prefer to get such notifications about content updates being revoked at the time it is revoked, not some days later.

[acc6d0b3610eec313831f7900fdbd235]

Hi @Anon1 @Brandon_Wertz @SuryaR

Palo Alto published a Dynamic Content Update issue announced last Thursday with Content Release 709.

https://live.paloaltonetworks.com/t5/Customer-Advisories/UPDATED-Important-information-regarding-Con...

I would like to remind you of the following vendor’s best practices.

1. For regular content update that comes out every Tuesday, we should recommend that customers wait a minimum of 24-48 hours before rolling it out to their environment. This is because with any content update, a bug may be introduced that will break production traffic (false positive). Waiting for at least 24-48 hours allows our Content Team to notify customers of any potential issue before they are impacted.
   
   
2. For emergency content update, we should recommend customers to roll it out asap because it contains threat signature for zero day exploits. Please be aware that all Palo Alto content updates are cumulative (i.e. emergency update 123 = regular update 122 + new emergency signature). So by installing it you will also install the previous regular update. In some cases, this may void the 24-48 hour delay recommendation above. But in this case the risk of getting hit by zero day exploit outweighs potential app ID/signature false positive.
   
   
3. At the end of the day, we should try to help customers strike a reasonable balance between securing networks and minimizing outages.

Below is a good configuration example:

From PANOS UI, go to Devices > Dynamic Updates > Application and Threats. In this case we are asking firewall to download and install content update every day at 2AM but with threshold (wait period) of 24 hours. So no matter when a content update is released, we will always wait 24 hours before installation. You can adjust threshold to your requirement.

I hope this helps.

[SuryaR]

Hi @acc6d0b3610eec313831f7900fdbd235

Thank you for detailed explanation. Understood about the recommendations. 

My question:

How is this feasible in an enviroment if I have 50+(not including passive pair) firewalls .

I also thought panorama might be of help here. But in case, if a update is revoked and you try to push dynamic updates via panorama, it fails horribly.

For example:

We have dynamic updates set  to check daily, with a threshold of 8 hrs.

709 was relased( threshold was set to 8 hrs ). After 8 hrs 709 is installed on every firewall.

~14 hrs later PA decides to revoke 709.

Next, all I can think of is panorama, I do check updates (I find 708 is latest/available). Awesome, lets try now to push 708 to all firewalls,.

Panorama complains, the firewalls have better version than what it am trying to push. 

EPIC Fail...!!!

Only thing I can do is to Login to every firewall and push it back to 708 or wait for 24 hrs before the firewall updates itself. 

My 0.02$  is PA should make emergency and regular updates different in the dynamic updates tab, rather than combining them and pushing them once, just how checkpoint does.

Hope I made sense. 

[acc6d0b3610eec313831f7900fdbd235]

@SuryaR

I agree with your suggestion. This would most likely be a feature request or something that they may be working in the background to get it fixed, since it is not the first time it happens. In fact this it the 3rd issue since last year. 2 in the past 2 weeks, but this issues happend with every vendor unfortunately.

The way I am dealing with it currently using Panorama, is creating an update schedule in Panorama itself. This takes care of the issue when you have multiple firewall like you described. Device Deployment --> Dynamic Updates --> Schedules. This way you can centrally control from Panorama directly which and when the updates should be pushed to the firewalls.

As I mentioned previously, best practices is to set the threshold for 24 or 48 hours after the update is released, so that if an issue with the signature occurs, you are safe for a little while, until they can release the correct signature.

I hope this helps.

[Anon1]

I concur with SuryaR´s suggestion to differentiate between normal and emergency updates. With long threshold times like 48h it can happen that the device is waiting and waiting for the threshold to be reached.

Normal content update released => do not install until 48h old

Within 48h, emergency content update released => do not install until 48h old

etc.

Rolling out content updates manually thwarts the ability to prevent known attacks automatically.