Query on deleting incidents

[assubramania]

Hi Team,

The standard customer has a query regarding deleting an incident on XSOAR. 1. In terms of performance usage when delete an incident or multiple selected incidents on the XSOAR console (Incidents page) and when delete an incident or multiple selected incidents through API call on endpoint POST /incident/batchDelete. The question is, which option is consume a better performance when do with the same number of incident records? 2. The maximum number of incident records can select and delete on XSOAR console incident page at one time? 3. The maximum number of incident records can select and delete via API call on endpoint /incident/batchDelete at one time?

We provided an update as below:

All these questions depend on the resources allocated on your server, your bandwidth, etc. Generally, it is better to delete the incidents using the API as you get rid of the browser consuming resources and loading the pages. Regarding the limit of incidents, there is no documented limit because it also depends on your XSOAR performance, we recommend to delete the incidents in batches of 50 from the UI as it is the number of incidents shown by default. You can increase this number using the API.

Now the customer requirement is: 

"You can increase this number using the API. "

Could you provide the maximum number of iocs to be removed via BulkAPI?

Is this right " The maximum number of Indicators of Compromise (IOCs) that can be removed via the Bulk API in a single request is 10,000. This limit helps ensure efficient processing and prevents overloading the system". Please confirm and direct if we have any documents.

[ctopay · Accepted solution]

Starting with incidents, yes it's not documented to that extent, but here are some known facts.

- Technically UI vs API is not that different, but UI load and refresh takes time. And if you try to delete lots(hundreds) of incidents by selecting all, you may experience timeouts.

- For /incident/batchDelete on the latest version(6.12) the default limit is 100 incidents afaik. However, you can raise this in the post request using the page/size keys. (For example: {"page":0,"size":batch_size,"query":"some_query", "lastSeen": {"fromDate":"2022-07-16T22:00:00Z"})  So, it requires a programmatic approach to use with paging in loops.

- There is this community pack: https://cortex.marketplace.pan.dev/marketplace/details/CommunityCommonScripts/

and there is an automation called DeleteIncidentsByQuery in it, which uses Core-API >'core-delete-incidents'. Again it can be used in a loop, command has a size parameter.

- Finally: We always advise archiving when a customer wants to get rid of lots of incidents & data. When possible use: https://docs-cortex.paloaltonetworks.com/r/Cortex-XSOAR/6.12/Cortex-XSOAR-Administrator-Guide/Free-u...

For Indicators, I don't know an automation called BulkApi. But /indicators/batchDelete is almost the same as incidents. The default limit should be again 100(never tested) and you need to programmatically process higher batches using the page/size keys in a loop creating new pages with the limit of 100 iocs/iteration. (For example: {"page": 0, "size": batch_size, "query": "some_query","sort": [{"field": "calculatedTime","asc": False}]})

But remember that, retention setting is also a way to manage indicator lifecycle(when possible).

[joramirez]

Additionally,in order to get the correct data to delete, then you compose a query. First create a dashboard and download the json where it has the query inside. This way you can test first in the dashboard what are the list of incidents you want to delete.