topic Re: Integrate with MISP in General Topics

Integrate with MISP

SantiBT — Mon, 20 Feb 2017 17:02:32 GMT

Hi all,

Do you know something sample about integration with MISP (Malware Information share platform)???

So another question is about scripts, can I launch a script into conifg a new prototype? If I've created a new prototype I set a url option...can I set the url option for script option????

Thanks a lot

Re: Integrate with MISP

lmori — Tue, 21 Feb 2017 18:20:01 GMT

Hi @SantiBT,

about MISP integration, we are planning to add it in the short term. Would you be interested in a Miner for MISP or sending indicators to MISP ?

luigi

Re: Integrate with MISP

SantiBT — Wed, 22 Feb 2017 08:26:11 GMT

Hi Lmori!!

Yes, I'm interesting in a miner for MISP!!!! it will be a great idea!!!!

Do you known that??????

Please, let me know if you need more info about this!!

Regards!

Re: Integrate with MISP

lmori — Wed, 22 Feb 2017 17:36:49 GMT

Hi @SantiBT,

I am planning to start working on it in a couple of weeks, would you be interested in testing the beta ?

luigi

Re: Integrate with MISP

SantiBT — Thu, 23 Feb 2017 08:21:48 GMT

Of course! Tell me when and I'll check your mine!

Thanks a lot

Re: Integrate with MISP

mateusz_o — Mon, 27 Mar 2017 09:37:15 GMT

Hi,

maybe you already have some beta version for testing?

Re: Integrate with MISP

lmori — Mon, 27 Mar 2017 11:45:23 GMT

Sorry, running late on this. First beta code should be available the week of April 10th (2017)

Re: Integrate with MISP

Sebastiano — Wed, 23 Aug 2017 07:51:26 GMT

Hi everyone,

I succeeded in using MISP extension in order to get data from a misp server...but now I cannot

export data via output node.

My feed pass through a stdlib.aggregatorDomain and then I'm trying to have them available through a stdlib.feedLCGreenWithValue output node.

No luck so far...on the output node I see non zero statistics for

updated.queue, update.rx, withdraw.processed, withdraw.queued, withdraw.rx

while zero value for

checkpoint.* and removed

If I try to connect to the FEED BASE URL of the output node I get status 200 but a blank page.

I'm probably overlooking some important point...

Regards.

Sebastiano

Re: Integrate with MISP

lmori — Wed, 23 Aug 2017 14:20:14 GMT

Hi @Sebastiano,

could you check in the Miner LOGS which type of share level is applied to the indicators ?

Go to the MISP Miner and click on LOGS, there you will see the extracted indicators. If you click on one of them you will see the full list of attributes assigned to the indicators and you will be able to check the share_level attribute.

Re: Integrate with MISP

Sebastiano — Thu, 24 Aug 2017 09:00:10 GMT

Hi @lmori and thanks a lot for you answer.

I checked the log of the misp miner and I see share_level set to 'white' so I think those should be good 'candidates' for output.

{
"_age_out": 4294967295000,
"confidence": 70,
"share_leve": "white",
"misp_event_tags": [

<snip>

}

I'm using minemeld version 0.9.40

and minemeld-misp version 0.1b5

kind regars

Seba

Edit....

Was a confidence problem...as my output node was a low confidence one... so confidence < 50...

Now it works like a charm...

Re: Integrate with MISP

vedd3r — Thu, 24 Aug 2017 09:16:02 GMT

Is it possible to create one for sending indicators to MISP as well? Would be great if it can work both ways. Reason is that, MineMeld can take a lot of indicators from different sources, which some of them will create a lot of noise/false positives (IPv4 for example) and need to be 'curated' and 'enriched' before feeding it to other platforms such as SIEMs. So from my perspective MISP fits into that role of repository and enricher. Also, GOSINT from Cisco looks promising as well when it comes to data enricher.

Re: Integrate with MISP

xhoms — Thu, 24 Aug 2017 12:50:39 GMT

@vedd3r,

MineMeld is modular enough to accomodate 'enrichement'. For instance you could create an aggregator sort of node that checks IPv4 against your threat intel source (i.e. Wildfire / AutoFocus) to attach 'enrichement attributes' to that indicator.

I'm planning, for instance, on creating an enrichement node for MineMeld that will attach the PAN-DB URL category value as an attribute to all URL indicators received by that node. Each node in a MineMeld graph has the native capability of filtering (accept/discard) indicators based on attribute values. In my case the idea will be for the output node to discard all URL indicators received from this 'enriched graph' that are classified as malware or phishing by PAN-DB because the URL-Filtering feature would be taking care of them already.

Re: Integrate with MISP

vedd3r — Sun, 03 Sep 2017 13:36:43 GMT

Thanks

@xhoms wrote:
@vedd3r,

MineMeld is modular enough to accomodate 'enrichement'. For instance you could create an aggregator sort of node that checks IPv4 against your threat intel source (i.e. Wildfire / AutoFocus) to attach 'enrichement attributes' to that indicator.

I'm planning, for instance, on creating an enrichement node for MineMeld that will attach the PAN-DB URL category value as an attribute to all URL indicators received by that node. Each node in a MineMeld graph has the native capability of filtering (accept/discard) indicators based on attribute values. In my case the idea will be for the output node to discard all URL indicators received from this 'enriched graph' that are classified as malware or phishing by PAN-DB because the URL-Filtering feature would be taking care of them already.

Thanks!

Re: Integrate with MISP

iThreatHunt — Wed, 27 Sep 2017 10:10:26 GMT

I found error when activate "minemeld-misp" extensions

Please recommend me:

Collecting pymisp (from minemeld-misp==0.1b5)
/opt/minemeld/engine/0.9.42/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/opt/minemeld/engine/0.9.42/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /simple/pymisp/
/opt/minemeld/engine/0.9.42/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning

see full log in attachments

Re: Integrate with MISP

iThreatHunt — Fri, 03 Nov 2017 10:05:00 GMT

@lmori

Do you have .yml file? My company block .git file from Server.

https://github.com/PaloAltoNetworks/minemeld-misp

Re: Integrate with MISP

lmori — Thu, 09 Nov 2017 08:58:18 GMT

Hi @iThreatHunt,

I have built a wheel file for it, you can download it and upload to MineMeld:

https://github.com/PaloAltoNetworks/minemeld-misp/releases/download/0.1b5/minemeld_misp-0.1b5-py2-none-any.whl

Re: Integrate with MISP

iThreatHunt — Thu, 09 Nov 2017 09:16:29 GMT

@lmori

Thank you. But I activate API. I found message:

Processing /opt/minemeld/local/library/minemeld_misp-0.1b5-py2-none-any.whl
Requirement already satisfied: minemeld-core==0.9.44 in /opt/minemeld/engine/0.9.44/lib/python2.7/site-packages (from -c /opt/minemeld/local/library/constraints.txt (line 31))
Collecting pymisp (from minemeld-misp==0.1b5)
/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /simple/pymisp/
/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /simple/pymisp/
/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /simple/pymisp/
/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /simple/pymisp/
/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /simple/pymisp/
/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Could not find a version that satisfies the requirement pymisp (from minemeld-misp==0.1b5) (from versions: )
No matching distribution found for pymisp (from minemeld-misp==0.1b5)
/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning

Re: Integrate with MISP

lmori — Tue, 14 Nov 2017 05:28:30 GMT

Hi @iThreatHunt,

are you restricting access from the VM to Internet ? It seems that python pip library is not being to reach out to the package servers.

Thanks,

luigi

Re: Integrate with MISP

iThreatHunt — Tue, 14 Nov 2017 07:02:35 GMT

Yes, I allow access to Internet by a URL for MM Server. Could you recommend me for URL list that must allow it?

Re: Integrate with MISP

iThreatHunt — Wed, 17 Jan 2018 08:45:50 GMT

SSL Warnings

urllib3 will issue several different warnings based on the level of certificate verification support. These warning indicate particular situations and can resolved in different ways.

InsecureRequestWarning

This happens when an request is made to an HTTPS URL without certificate verification enabled. Follow the certificate verification guide to resolve this warning.
InsecurePlatformWarning

This happens on Python 2 platforms that have an outdated ssl module. These older ssl modules can cause some insecure requests to succeed where they should fail and secure requests to fail where they should succeed. Follow the pyOpenSSL guide to resolve this warning.

SNIMissingWarning

This happens on Python 2 versions older than 2.7.9. These older versions lack SNI support. This can cause servers to present a certificate that the client thinks is invalid. Follow the pyOpenSSL guide to resolve this warning.