Another week, another BTP quirk: Behavioral threat detected (rule: sync.enable_safemode_on_next_reboot) spawned from VEEAM

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements

Another week, another BTP quirk: Behavioral threat detected (rule: sync.enable_safemode_on_next_reboot) spawned from VEEAM

We are observing VEEAM VeeamTransportSvc.exe being blocked by BTP and, thus, preventing backups from being started.


We are working on a temporary fix excluding path and cgo and the likes but this is the second week in a row that content updates are screwing, this time impacting operations.

 

Already filled a support case.

 

BR

55 REPLIES 55

L1 Bithead

My DC backups usually take less than 2 minutes each so I have contemplated trying to disable the tool for that time and run the backup.

"c:\program files\palo alto networks\traps\cytool.exe" protect disable

then after the backup
"c:\program files\palo alto networks\traps\cytool.exe" protect enable

 

That should work according to what I've read as a stop gap until cortex tram figures out what caused the false trigger. 

L0 Member

Hello!

 

The best way to prevent this behavior from being blocked in my opinion is to create an alert exception:

  1. Right click the alert
  2. Click "Manage Alert"
  3. Click "Create Alert Exception"
    CSGandD_1-1660752639087.png
  4. Check the options "CGO Process Path" and then "CGO Command Arguments"
    CSGandD_2-1660752772341.png

     

  5. Select the desired scope and then click "Create"

Please note that this Behavioral Threat is an attempt by Palo Alto to detect the exploitation of recently released boot vulnerabilities:

Edit: This detection may more specifically relate to detecting boot configuration changes used by ransomware groups, that said, I can imagine both techniques being used in tandem:

 

The only downside to this is that there would now be an exception and it could be exploited by a legit threat.

You are absolutely correct. Ideally, if the block is not impacting, it shouldn't be removed.

  • If it is impacting, the vulnerabilities should be mitigated prior to removing the block if possible
  • If they cannot be mitigated without removing the block:
    • Disconnect the host from the network
    • Remove the block
    • Apply the available patch
  • And finally, if the block needs to be removed AND host cannot be disconnected from the network due to essential services being impacted:
    • Cross your fingers, close your eyes, remove the block and then apply the patch if this is within your risk tolerances, lol

Best of luck to all, hopefully more targeted indicators of the actual exploits are identified soon!

 

L0 Member

Do we know for sure that even if the referenced windows update in those articles (below) gets installed, that the Cortex BTP no longer affects the system?  If it still does affect things, then we need a better answer than to whitelist a critical component of Windows that plenty of malware targets.  

Isn't there a way to do a BTP exception within BTP somehow vs global exception of svchost.exe?

KB5012170 

I ran these scripts and then was able to backup our domain controllers again. I rebooted one just to see what happened and the Veeam error came back again after the reboot.

I tried disabling Cortex on a DC via cytool as I mentioned earlier but the backups still failed.

Interesting. Well hopefully this will help support narrow in on the root cause!

For real! If you run the scripts again after a reboot you will be to run backups again.

Ok so the workaround is to re-register the vss components and don't reboot, backups should run as expected.

For the resolution, I think that a quick workaround could be check the value to which the reg key is setted, if is not one that cause a safeboot, raise an alert but not block the execution.

 

Post DEFCON shenanigans at it's best, seems to me..

L1 Bithead

On a hunch, rather than re-registering all the files, I just plucked out the service names and did a stop/start on each of them. 
Next I tried the backup and it worked. Having said that, it looks like the regsvr32 portion of this is not required.
Also, the stop/start didn't work automatically on all of them. I ran them one at a time and some said they weren't even running. Some also stopped other services. After trying to stop/start them all, I sorted the services by startup type and manually started any that weren't restarted from my stop/start. After I did that, I tried the backup and success.

L1 Bithead

Here's what got my backup working. I added the necessary services here that the original script was not restarting.

net stop "System Event Notification Service" /y
net start "System Event Notification Service"
net stop "Background Intelligent Transfer Service" /y
net start "Background Intelligent Transfer Service"
net stop "COM+ Event System" /y
net start "COM+ Event System"
net stop "Microsoft Software Shadow Copy Provider" /y
net start "Microsoft Software Shadow Copy Provider"
net stop "Volume Shadow Copy"
net start "Volume Shadow Copy"

net start "System Event Notification Service"
net start "DFS Replication"

Got a ticket reply this early morning, FYI for everyone that had not opened a ticket

Comment: Hi Roberto,

We have released a KB article noting on this issue with Veeam Backup, for your reference,

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA14u000000wlJGC

The Eng Team has deployed the fix in CU650-11590, please make sure affected endpoints are updated with this CU to avoid the prevention of Veeam Backup process.

We apologise for any inconvenience caused.

Regards,

 

 

Note: the kb article, at this moment, leads me to a salesforce error page

Yes, I have the same problem with the salesforce error page. 

  • 25289 Views
  • 55 replies
  • 10 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!