topic Re: Ingest Taxii feed into XSOAR 6.12 in Cortex XSOAR Discussions

Ingest Taxii feed into XSOAR 6.12

TonyZhu — Mon, 02 Oct 2023 18:34:38 GMT

Hi,

I am trying to ingest our taxii feed into XSOAR 6.12 with following steps:

installed XSOAR 6.12 on ubuntu 22.0.4 LTS
launched the web portal, and installed TAXII Feed (1.x) pack from marketplace
Ingest feed using "Integration Instance Settings"
- Typed in the parameters such as name, discovery service URL, username/password, collection name, poll service url, first fetch time,set Feed Fetch Interval to 10 mins, etc.
- Test successful

With above steps, it was able to pull indicator from the collection I specified, but, it seems every time it only pulls one indicator and the same one over and overall again, the taxii feed provides over thousands of indicators per day, but I only see one indicator on Threat Intel dashboard -> XSOAR Indicators.

Note, I have also tested the same feeds with other platforms such as ThreatQ and ThreatConnect, from there the feeds are ingested as expected.

Could someone please advise on it?
# XSOAR6.12 #taxii integration

Re: Ingest Taxii feed into XSOAR 6.12

albmartinez — Tue, 03 Oct 2023 14:19:35 GMT

Hello @TonyZhu ,

Are you using the same discovery service and collection for the ThreatConnect and ThreatQ?

Also, what is your "First Fetch Time" set to?

Please advise,

Re: Ingest Taxii feed into XSOAR 6.12

TonyZhu — Tue, 03 Oct 2023 14:42:44 GMT

Hi @albmartinez,

Thank you for looking into it.

Yes. It's the same service and collection.

The "First Fetch Time" set to 1 day.

Re: Ingest Taxii feed into XSOAR 6.12

albmartinez — Tue, 03 Oct 2023 15:15:24 GMT

Hi Tony,

Try incrementing that setting to intervals of 5 days to see if the indicators increase.

It looks like it may be only looking at the feed if the indicators are day old but may have more indicators if your first fetch time is more days back.

Re: Ingest Taxii feed into XSOAR 6.12

TonyZhu — Tue, 03 Oct 2023 15:47:01 GMT

Thanks @albmartinez

It's the same behavior after setting it to 5 days...

Re: Ingest Taxii feed into XSOAR 6.12

TonyZhu — Wed, 04 Oct 2023 21:32:56 GMT

Hi @albmartinez,

Any more feedback would be appreciated. Thanks!

Re: Ingest Taxii feed into XSOAR 6.12

chrking — Mon, 16 Oct 2023 01:30:56 GMT

What does the fetch history (the anticlockwise arrow icon) on the instance say was fetched?

What query are you using on the threat intel page? If you change the query to something like "sourceInstance:<instance name>" do you get different results?

Re: Ingest Taxii feed into XSOAR 6.12

TonyZhu — Tue, 17 Oct 2023 20:18:02 GMT

Thanks @chrking for looking into it.

The fetch history shows that there are only 2 indicators pulled every scheduled job.

The query in threat intel page shows the same result.

Re: Ingest Taxii feed into XSOAR 6.12

chrking — Wed, 18 Oct 2023 01:59:58 GMT

OK, so this is actually really good info. The fact that the time isn't changing means that it's not updating it's last run timestamp for some reason. This shouldn't happen under normal operations. I'd suggest turning debug mode on, then looking at the output in your integration-instance.log. It's possible there are issues parsing the results that are causing the fetch to terminate early and not update the last run timestamp.

Re: Ingest Taxii feed into XSOAR 6.12

TonyZhu — Thu, 19 Oct 2023 21:05:13 GMT

Thanks @chrking

I checked the integration-instance.log under debug mode, there was no exceptions/error in it but I noticed that the timestamp is "created": "0001-01-01T00:00:00Z" in following logs, does it look right to you?

2023-10-19 08:36:22.8314 info (maliciousFile_TAXIIFeed_fetch-indicators) debug-mode started. #### http client print found: False. #### Env {'PATH': '/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'HOSTNAME': 'cd2c01af9608', 'HTTP_PROXY': '', 'http_proxy': '', 'HTTPS_PROXY': '', 'https_proxy': '', 'no_proxy': '', 'NO_PROXY': '', 'LANG': 'C.UTF-8', 'GPG_KEY': 'A035C8C19219BA821ECEA86B64E628F8D684696D', 'PYTHON_VERSION': '3.10.13', 'PYTHON_PIP_VERSION': '23.0.1', 'PYTHON_SETUPTOOLS_VERSION': '65.5.1', 'PYTHON_GET_PIP_URL': 'https://github.com/pypa/get-pip/raw/9af82b715db434abb94a0a6f3569f43e72157346/public/get-pip.py', 'PYTHON_GET_PIP_SHA256': '45a2bb8bf2bb5eff16fdd00faef6f29731831c7c59bd9fc2bf1f3bed511ff1fe', 'DOCKER_IMAGE': 'demisto/taxii:1.0.0.76522', 'HOME': '/root'}. #### Params: { "cert_text": null, "collection": "malicious-file", "credentials": { "credential": "", "credentials": { "cacheVersn": 0, "created": "0001-01-01T00:00:00Z", "id": "", "locked": false, "modified": "0001-01-01T00:00:00Z", "name": "", "password": "MASKED_SECRET", "sizeInBytes": 0, "sshkey": "", "sshkeyPass": "", "user": "", "vaultInstanceId": "", "version": 0, "workgroup": "" }, "identifier": "{user_name}", "password": "MASKED_SECRET", "passwordChanged": false }, "creds_certificate": null, "discovery_service": "https://{feed-url}/discovery", "feed": true, "feedBypassExclusionList": true, "feedExpirationInterval": 0, "feedExpirationPolicy": "never", "feedFetchInterval": 10, "feedReliability": "A - Completely reliable", "feedReputation": "Malicious", "feedTags": null, "initial_interval": "5 day", "insecure": false, "key_text": "", "poll_service": "https://{feed-service}/taxii11/poll", "polling_timeout": "20", "proxy": false, "subscription_id": null, "tlp_color": "RED" }.

Re: Ingest Taxii feed into XSOAR 6.12

TonyZhu — Thu, 19 Oct 2023 21:07:20 GMT

@chrking, the request and response in the log, it's keep pulling the same indicators over and over again, while the feed generates thousands new indicators everyday

'<taxii_11:Poll_Request xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1"\nmessage_id="e0509e9e-9a0c-4155-9987-6a7914a99cb1"\ncollection_name="malicious-file"\n>\n<taxii_11:Exclusive_Begin_Timestamp>2023-10-17T23:20:00Z</taxii_11:Exclusive_Begin_Timestamp>\n<taxii_11:Inclusive_End_Timestamp>2023-10-19T15:36:00Z</taxii_11:Inclusive_End_Timestamp>\n<taxii_11:Poll_Parameters allow_asynch="false"><taxii_11:Response_Type>FULL</taxii_11:Response_Type></taxii_11:Poll_Parameters>\n</taxii_11:Poll_Request>' (source: /builds/GOPATH/src/gitlab.xdr.pan.local/xdr/xsoar/server/services/automation/dockercoderunner.go:979) 2023-10-19 08:37:11.5815 info (maliciousFile_TAXIIFeed_fetch-indicators) cURL: curl -X POST https://{feed-service}/taxii11/poll -H "Accept: */*" -H "Content-Type: application/xml" -H "X-TAXII-Content-Type: urn:taxii.mitre.org:message:xml:1.1" -H "X-TAXII-Accept: urn:taxii.mitre.org:message:xml:1.1" -H "X-TAXII-Services: urn:taxii.mitre.org:services:1.1" -H "X-TAXII-Protocol: urn:taxii.mitre.org:protocol:https:1.0" -H "Authorization: Basic <XX_REPLACED>" --noproxy "*" -d '<taxii_11:Poll_Fulfillment xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1"\n message_id="a6a2a4aa-5354-4c3a-adec-2ab50105add4" collection_name="malicious-file" result_id="d257e14a-76bc-40bd-a6f8-af2cea5778f5"\n result_part_number="2"/>' (source: /builds/GOPATH/src/gitlab.xdr.pan.local/xdr/xsoar/server/services/automation/dockercoderunner.go:979) 2023-10-19 08:37:11.5836 info (maliciousFile_TAXIIFeed_fetch-indicators) cURL: curl -X POST https://{feed-service}/taxii11/poll -H "Accept: */*" -H "Content-Type: application/xml" -H "X-TAXII-Content-Type: urn:taxii.mitre.org:message:xml:1.1" -H "X-TAXII-Accept: urn:taxii.mitre.org:message:xml:1.1" -H "X-TAXII-Services: urn:taxii.mitre.org:services:1.1" -H "X-TAXII-Protocol: urn:taxii.mitre.org:protocol:https:1.0" -H "Authorization: Basic <XX_REPLACED>" --noproxy "*" -d '<taxii_11:Poll_Fulfillment xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1"\n message_id="71a32d1c-6c6f-4480-b10e-253bdc76f2b1" collection_name="malicious-file" result_id="d257e14a-76bc-40bd-a6f8-af2cea5778f5"\n result_part_number="3"/>' (source: /builds/GOPATH/src/gitlab.xdr.pan.local/xdr/xsoar/server/services/automation/dockercoderunner.go:979) 2023-10-19 08:37:11.5854 info (maliciousFile_TAXIIFeed_fetch-indicators) cURL: curl -X POST https://{feed-service}/taxii11/poll -H "Accept: */*" -H "Content-Type: application/xml" -H "X-TAXII-Content-Type: urn:taxii.mitre.org:message:xml:1.1" -H "X-TAXII-Accept: urn:taxii.mitre.org:message:xml:1.1" -H "X-TAXII-Services: urn:taxii.mitre.org:services:1.1" -H "X-TAXII-Protocol: urn:taxii.mitre.org:protocol:https:1.0" -H "Authorization: Basic <XX_REPLACED>" --noproxy "*" -d '<taxii_11:Poll_Fulfillment xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1"\n message_id="1a058a73-0682-4776-bda7-20b02e2e330a" collection_name="malicious-file" result_id="d257e14a-76bc-40bd-a6f8-af2cea5778f5"\n result_part_number="4"/>' (source: /builds/GOPATH/src/gitlab.xdr.pan.local/xdr/xsoar/server/services/automation/dockercoderunner.go:979)

Re: Ingest Taxii feed into XSOAR 6.12

chrking — Tue, 24 Oct 2023 07:11:31 GMT

@TonyZhu the created/modified dates here are related to the credential I think, this likely just means you're using a username/password configured in the integration itself rather than a linked credential and isn't inherently concerning.

These parts pulled out from the logs look like requests rather than responses, but it's interesting that XSOAR is pulling 4 full pages of results but only returning 2 results.

This definitely looks like some kind of incompatibility between the way your taxii server is returning the results and the way XSOAR is parsing them. I'd love to set up my own version of the taxii client with additional debugging details so I can see exactly what is being pulled, but I'm kind of guessing this is non-public threat intel?

Would you be able to SSH to your XSOAR server, execute one of the curl commands from that file with the <XX_REPLACED> part restored to valid basic auth (see https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Authorization#basic_authentication ) and post a sample of the results showing a redacted version of an un-fetched indicator? I'm looking for the XML structure of the indicator rather than any content, so feel free to replace any actual content with "REDACTED".

Filing a support case would be the other option, that way you could share the results without it being public.

Re: Ingest Taxii feed into XSOAR 6.12

TonyZhu — Thu, 02 Nov 2023 15:11:44 GMT

Thanks @chrking.

This is the response of the cabby-client request from malicious-url collection, it returns malicious uri/IP indicators in STIX format that is the same as curl commands returns. I truncate the result to a few indicators from each pages (there are 3 pages in total):

We also have other collections such as malicious-file that contains file hash indicators, please let me know if you want to look at it's response that is similar format to malicious-uri:

<stix:STIX_Package xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:indicator="http://stix.mitre.org/Indicator-2" xmlns:cybox="http://cybox.mitre.org/cybox-2" xmlns:cyboxCommon="http://cybox.mitre.org/common-2" xmlns:URIObject="http://cybox.mitre.org/objects#URIObject-2" xmlns:DomainNameObj="http://cybox.mitre.org/objects#DomainNameObject-1" xmlns:FileObj="http://cybox.mitre.org/objects#FileObject-2" xmlns:AddressObject="http://cybox.mitre.org/objects#AddressObject-2" xmlns:report="http://stix.mitre.org/Report-1" xmlns:threat-actor="http://stix.mitre.org/ThreatActor-1" xmlns:ttp="http://stix.mitre.org/TTP-1" xmlns:stix-ciq="http://stix.mitre.org/extensions/Identity#CIQIdentity3.0-1" xmlns:stixCommon="http://stix.mitre.org/common-1" xmlns:xal="urn:oasis:names:tc:ciq:xal:3" xmlns:xpil="urn:oasis:names:tc:ciq:xpil:3" xmlns:cyboxVocabs="http://cybox.mitre.org/default_vocabularies-2" xmlns:stixVocabs="http://stix.mitre.org/default_vocabularies-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" id="<REDACTED>-threat-intel:package-d9178671-1c06-4450-b9b1-cc23ccbd191d" timestamp="2023-10-15T00:00:00Z" version="1.2"> <stix:STIX_Header> <stix:Title>malicious-uri for 2023-10-15T00:00:00Z - Page:1</stix:Title> <stix:Description>malicious-uri for 2023-10-15T00:00:00Z - Page:1</stix:Description> </stix:STIX_Header> <stix:Observables cybox_major_version="2" cybox_minor_version="1"> <cybox:Observable id="<REDACTED>-threat-intel:observable-3cb1a50b-0e2f-406b-8c7c-b2b7027027b8"> <cybox:Object id="<REDACTED>-threat-intel:URI-c966274b-a11f-4971-890d-739a661a34ad"> <cybox:Properties xsi:type="DomainNameObj:DomainNameObjectType" type="FQDN"> <DomainNameObj:Value><REDACTED phishing.com></DomainNameObj:Value> <cyboxCommon:Custom_Properties> <cyboxCommon:Property name="confidence">100</cyboxCommon:Property> <cyboxCommon:Property name="categories">Phishing</cyboxCommon:Property> </cyboxCommon:Custom_Properties> </cybox:Properties> </cybox:Object> </cybox:Observable> ... <cybox:Observable id="<REDACTED>-threat-intel:observable-190320dd-e5b4-44f8-be8b-79c5e84ef4e5"> <cybox:Object id="<REDACTED>-threat-intel:URI-621605c7-0f06-4ac9-8609-60ff5a533226"> <cybox:Properties xsi:type="DomainNameObj:DomainNameObjectType" type="FQDN"> <DomainNameObj:Value><REDACTED malicious.com></DomainNameObj:Value> <cyboxCommon:Custom_Properties> <cyboxCommon:Property name="confidence">100</cyboxCommon:Property> <cyboxCommon:Property name="categories">Malicious Outbound Data/Botnets</cyboxCommon:Property> </cyboxCommon:Custom_Properties> </cybox:Properties> </cybox:Object> </cybox:Observable> </stix:Observables> </stix:STIX_Package> <stix:STIX_Package xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:indicator="http://stix.mitre.org/Indicator-2" xmlns:cybox="http://cybox.mitre.org/cybox-2" xmlns:cyboxCommon="http://cybox.mitre.org/common-2" xmlns:URIObject="http://cybox.mitre.org/objects#URIObject-2" xmlns:DomainNameObj="http://cybox.mitre.org/objects#DomainNameObject-1" xmlns:FileObj="http://cybox.mitre.org/objects#FileObject-2" xmlns:AddressObject="http://cybox.mitre.org/objects#AddressObject-2" xmlns:report="http://stix.mitre.org/Report-1" xmlns:threat-actor="http://stix.mitre.org/ThreatActor-1" xmlns:ttp="http://stix.mitre.org/TTP-1" xmlns:stix-ciq="http://stix.mitre.org/extensions/Identity#CIQIdentity3.0-1" xmlns:stixCommon="http://stix.mitre.org/common-1" xmlns:xal="urn:oasis:names:tc:ciq:xal:3" xmlns:xpil="urn:oasis:names:tc:ciq:xpil:3" xmlns:cyboxVocabs="http://cybox.mitre.org/default_vocabularies-2" xmlns:stixVocabs="http://stix.mitre.org/default_vocabularies-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" id="<REDACTED>-threat-intel:package-ae0e2b3f-e030-4138-a969-f63b7e13b700" timestamp="2023-10-15T00:00:00Z" version="1.2"> <stix:STIX_Header> <stix:Title>malicious-uri for 2023-10-15T00:00:00Z - Page:2</stix:Title> <stix:Description>malicious-uri for 2023-10-15T00:00:00Z - Page:2</stix:Description> </stix:STIX_Header> <stix:Observables cybox_major_version="2" cybox_minor_version="1"> <cybox:Observable id="<REDACTED>-threat-intel:observable-98dc9483-a600-462f-8524-611b73bfff0a"> <cybox:Object id="<REDACTED>-threat-intel:URI-ddf796cb-082b-4e69-b536-5f379800e238"> <cybox:Properties xsi:type="DomainNameObj:DomainNameObjectType" type="FQDN"> <DomainNameObj:Value><REDACTED phishing.com></DomainNameObj:Value> <cyboxCommon:Custom_Properties> <cyboxCommon:Property name="confidence">100</cyboxCommon:Property> <cyboxCommon:Property name="categories">Phishing</cyboxCommon:Property> </cyboxCommon:Custom_Properties> </cybox:Properties> </cybox:Object> </cybox:Observable> ... <cybox:Observable id="<REDACTED>-threat-intel:observable-871f5a4b-9ede-46cd-811f-757bacd1ab7e"> <cybox:Object id="<REDACTED>-threat-intel:URI-7b4aee1c-53a1-429c-9c7d-b5777e14fa71"> <cybox:Properties xsi:type="DomainNameObj:DomainNameObjectType" type="FQDN"> <DomainNameObj:Value><REDACTED suspicious.com></DomainNameObj:Value> <cyboxCommon:Custom_Properties> <cyboxCommon:Property name="confidence">80</cyboxCommon:Property> <cyboxCommon:Property name="categories">Suspicious</cyboxCommon:Property> </cyboxCommon:Custom_Properties> </cybox:Properties> </cybox:Object> </cybox:Observable> </stix:Observables> </stix:STIX_Package> <stix:STIX_Package xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:indicator="http://stix.mitre.org/Indicator-2" xmlns:cybox="http://cybox.mitre.org/cybox-2" xmlns:cyboxCommon="http://cybox.mitre.org/common-2" xmlns:URIObject="http://cybox.mitre.org/objects#URIObject-2" xmlns:DomainNameObj="http://cybox.mitre.org/objects#DomainNameObject-1" xmlns:FileObj="http://cybox.mitre.org/objects#FileObject-2" xmlns:AddressObject="http://cybox.mitre.org/objects#AddressObject-2" xmlns:report="http://stix.mitre.org/Report-1" xmlns:threat-actor="http://stix.mitre.org/ThreatActor-1" xmlns:ttp="http://stix.mitre.org/TTP-1" xmlns:stix-ciq="http://stix.mitre.org/extensions/Identity#CIQIdentity3.0-1" xmlns:stixCommon="http://stix.mitre.org/common-1" xmlns:xal="urn:oasis:names:tc:ciq:xal:3" xmlns:xpil="urn:oasis:names:tc:ciq:xpil:3" xmlns:cyboxVocabs="http://cybox.mitre.org/default_vocabularies-2" xmlns:stixVocabs="http://stix.mitre.org/default_vocabularies-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" id="<REDACTED>-threat-intel:package-cce78e5a-29b6-4787-87d8-e55eb4592a2b" timestamp="2023-10-15T00:00:00Z" version="1.2"> <stix:STIX_Header> <stix:Title>malicious-uri for 2023-10-15T00:00:00Z - Page:3</stix:Title> <stix:Description>malicious-uri for 2023-10-15T00:00:00Z - Page:3</stix:Description> </stix:STIX_Header> <stix:Observables cybox_major_version="2" cybox_minor_version="1"> <cybox:Observable id="<REDACTED>-threat-intel:observable-9b4e5963-de78-4a06-a1b2-1c2fe4513ccc"> <cybox:Object id="<REDACTED>-threat-intel:URI-0e912113-6b00-4f6c-8333-fd5dbc07fe61"> <cybox:Properties xsi:type="DomainNameObj:DomainNameObjectType" type="FQDN"> <DomainNameObj:Value><REDACTED suspicious.com></DomainNameObj:Value> <cyboxCommon:Custom_Properties> <cyboxCommon:Property name="confidence">80</cyboxCommon:Property> <cyboxCommon:Property name="categories">Suspicious</cyboxCommon:Property> </cyboxCommon:Custom_Properties> </cybox:Properties> </cybox:Object> </cybox:Observable> <cybox:Observable id="<REDACTED>-threat-intel:observable-81ec8944-054d-4799-aefc-eaa45aa2ad17"> <cybox:Object id="<REDACTED>-threat-intel:URI-e33646db-faab-4815-a932-22eb1e7a0062"> <cybox:Properties xsi:type="URIObject:URIObjectType"> <URIObject:Value><REDACTED suspicious.com></URIObject:Value> <cyboxCommon:Custom_Properties> <cyboxCommon:Property name="confidence">80</cyboxCommon:Property> <cyboxCommon:Property name="port">58204</cyboxCommon:Property> <cyboxCommon:Property name="categories">Computer/Information Security,Suspicious</cyboxCommon:Property> </cyboxCommon:Custom_Properties> </cybox:Properties> </cybox:Object> </cybox:Observable> ... </cybox:Observable> <cybox:Observable id="<REDACTED>-threat-intel:observable-6019be08-2c42-40db-8ccc-55f46c9c856d"> <cybox:Object id="<REDACTED>-threat-intel:URI-8bc4bee3-3298-4af2-acb0-cd3e4a46dd23"> <cybox:Properties xsi:type="URIObject:URIObjectType"> <URIObject:Value><REDACTED suspicious.com></URIObject:Value> <cyboxCommon:Custom_Properties> <cyboxCommon:Property name="confidence">80</cyboxCommon:Property> <cyboxCommon:Property name="port">80</cyboxCommon:Property> <cyboxCommon:Property name="categories">Suspicious</cyboxCommon:Property> </cyboxCommon:Custom_Properties> </cybox:Properties> </cybox:Object> </cybox:Observable> </stix:Observables> </stix:STIX_Package>

Re: Ingest Taxii feed into XSOAR 6.12

chrking — Mon, 06 Nov 2023 04:35:57 GMT

@TonyZhu

Looking at the samples you've provided, these appear to be STIX packages rather than Poll Responses, which is what we'd be expecting for the response to a TAXII poll request. In a Poll Response, we'd expect the content to be inside a <Content_Block> tag which doesn't appear to be happening here and I suspect that's why nothing is getting parsed.

You should be able to parse Stix Packages with !CreateIndicatorsFromSTIX or similar, but there's no integration to automatically fetch STIX packages from a specific URL. If you want to go this route (as opposed to seeing if you can convince your taxii server to produce actual taxii responses) then a regularly scheduled job calling !HttpV2 and !CreateIndicatorsFromSTIX would be one possible option.

Re: Ingest Taxii feed into XSOAR 6.12

TonyZhu — Fri, 10 Nov 2023 00:01:15 GMT

@chrking

Thanks for the response.

I am able to use curl command that is in the integration-instance.log and replaced Authorization: Basic <XX_REPLACED> with Basic Auth Header.

With following command line:

curl -X POST https://api.sep.securitycloud.symantec.com/v1/threat-intel/taxii11/poll -H "Accept: */*" -H "Content-Type: application/xml" -H "X-TAXII-Content-Type: urn:taxii.mitre.org:message:xml:1.1" -H "X-TAXII-Accept: urn:taxii.mitre.org:message:xml:1.1" -H "X-TAXII-Services: urn:taxii.mitre.org:services:1.1" -H "X-TAXII-Protocol: urn:taxii.mitre.org:protocol:https:1.0" -H "Authorization: Basic <XX_REPLACED>" --noproxy "*" -d '<taxii_11:Poll_Request xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" message_id="59dec59d-2e24-4d72-9344-705e5e258813" collection_name="malicious-file"> <taxii_11:Exclusive_Begin_Timestamp>2023-10-27T23:20:00Z</taxii_11:Exclusive_Begin_Timestamp> <taxii_11:Inclusive_End_Timestamp>2023-10-29T19:10:00Z</taxii_11:Inclusive_End_Timestamp> <taxii_11:Poll_Parameters allow_asynch="false"><taxii_11:Response_Type>FULL</taxii_11:Response_Type></taxii_11:Poll_Parameters> </taxii_11:Poll_Request>'

The Poll_Response looks like following, there are thousands of file hash indicators so I truncated them in STIX packages.

<taxii_11:Poll_Response xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" message_id="907893d2-4dd2-4e06-b18c-b4a04642b5c8" in_response_to="59dec59d-2e24-4d72-9344-705e5e258813" result_id="b2e8cc8c-1b6a-4485-917f-7c5de8313f1c" collection_name="malicious-file" more="true" result_part_number="1"><taxii_11:Record_Count partial_count="false">2000</taxii_11:Record_Count> <taxii_11:Content_Block> <taxii_11:Content_Binding binding_id="urn:stix.mitre.org:xml:1.2"/> <taxii_11:Content> <stix:STIX_Package xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:indicator="http://stix.mitre.org/Indicator-2" xmlns:cybox="http://cybox.mitre.org/cybox-2" xmlns:cyboxCommon="http://cybox.mitre.org/common-2" xmlns:URIObject="http://cybox.mitre.org/objects#URIObject-2" xmlns:DomainNameObj="http://cybox.mitre.org/objects#DomainNameObject-1" xmlns:FileObj="http://cybox.mitre.org/objects#FileObject-2" xmlns:AddressObject="http://cybox.mitre.org/objects#AddressObject-2" xmlns:report="http://stix.mitre.org/Report-1" xmlns:threat-actor="http://stix.mitre.org/ThreatActor-1" xmlns:ttp="http://stix.mitre.org/TTP-1" xmlns:stix-ciq="http://stix.mitre.org/extensions/Identity#CIQIdentity3.0-1" xmlns:stixCommon="http://stix.mitre.org/common-1" xmlns:xal="urn:oasis:names:tc:ciq:xal:3" xmlns:xpil="urn:oasis:names:tc:ciq:xpil:3" xmlns:cyboxVocabs="http://cybox.mitre.org/default_vocabularies-2" xmlns:stixVocabs="http://stix.mitre.org/default_vocabularies-1" id="<REDACTED>:package-fa373751-d860-4fdf-8922-52739e01ca8d" timestamp="2023-10-27T23:20:00Z" version="1.2"> <stix:STIX_Header> <stix:Title>malicious-file for 2023-10-27T23:20:00Z - Page:1</stix:Title> <stix:Description>malicious-file for 2023-10-27T23:20:00Z - Page:1</stix:Description> </stix:STIX_Header> <stix:Observables cybox_major_version="2" cybox_minor_version="1"> <cybox:Observable id="<REDACTED>:observable-b2b52fb7-12e9-4051-9a7d-4807fd09c497"> <cybox:Object id="<REDACTED>:File-335ee5db-36ee-40ec-bc24-096f3d9ef21c"> <cybox:Properties xsi:type="FileObj:FileObjectType"> <FileObj:Hashes> <cyboxCommon:Hash> <cyboxCommon:Type xsi:type="cyboxVocabs:HashNameVocab-1.0">SHA256</cyboxCommon:Type> <cyboxCommon:Simple_Hash_Value><REDACTED></cyboxCommon:Simple_Hash_Value> </cyboxCommon:Hash> </FileObj:Hashes> </cybox:Properties> </cybox:Object> </cybox:Observable> <cybox:Observable id="<REDACTED>:observable-59d4a0f3-ae7a-426b-893c-d425d27e43f8"> <cybox:Object id="<REDACTED>:File-1627e197-09c2-47d8-9723-1bb1d37b05a2"> <cybox:Properties xsi:type="FileObj:FileObjectType"> <FileObj:Hashes> <cyboxCommon:Hash> <cyboxCommon:Type xsi:type="cyboxVocabs:HashNameVocab-1.0">SHA256</cyboxCommon:Type> <cyboxCommon:Simple_Hash_Value><REDACTED></cyboxCommon:Simple_Hash_Value> </cyboxCommon:Hash> </FileObj:Hashes> </cybox:Properties> </cybox:Object> </cybox:Observable> </stix:Observables> </stix:STIX_Package> </taxii_11:Content></taxii_11:Content_Block></taxii_11:Poll_Response>

Re: Ingest Taxii feed into XSOAR 6.12

chrking — Fri, 10 Nov 2023 06:25:35 GMT

@TonyZhu This first response only has a couple of indicators in it, but eyeballing them against the TAXII client code it seems like they should parse OK. At first I was wondering if XSOAR wasn't pulling subsequent pages of the poll response, but from the logs above it looks like it is.

I think you'll need Engineering and (probably) a custom debug version of the taxii client to troubleshoot this, sorry.

Re: Ingest Taxii feed into XSOAR 6.12

TonyZhu — Mon, 13 Nov 2023 16:57:55 GMT

Thanks @chrking. Really appreciate it!

There were lots of indicators (over thousands) in response I only kept few of them just for displaying, along with the response header.

Where I can get the customer debug version of the taxii client?