topic Re: Prototype for FS-ISAC in General Topics

Prototype for FS-ISAC

c_cong — Fri, 02 Jun 2017 12:56:23 GMT

I understand that Soltra is part of the existing 3rd party intelligence feed, just wondering has anyone created a prototype from FS-ISAC? THe portal address is https://portal.fsisac.com/

Understand from FS-ISAC, they uses Soltra as part of their intel too, is FS-ISAC intelligence pool as subset of Soltra?

Re: Prototype for FS-ISAC

lmori — Tue, 06 Jun 2017 07:49:49 GMT

Hi @c_cong,

from FS-ISAC you should retrieve the following data:

- URL of the TAXII discovery service

- name of the feed

- client certificate for authentication

On MineMeld:

- click on CONFIG and then on the hamburger icon to list the Prototypes

- click on hailataxii.guest_Abuse_ch and click on NEW

- modify the name of the new prototype

- under config copy & paste the following and change the feed name and the URL with the values you get from FS-ISAC:

age_out:
    default: last_seen+30d
    sudden_death: false
attributes:
    confidence: 30
    share_level: red
collection: <feedname>
discovery_service: <fs-isac discovery service>
source_name: fs-isac.<feedname>
client_cert_required: true

- press OK and then create a new node from the new prototype

- COMMIT

- after the engine has started, go in NODES click on the new NODE and upload the client certificate

Re: Prototype for FS-ISAC

iThreatHunt — Thu, 29 Jun 2017 10:30:13 GMT

Hi @lmori

I need to connect with FS-ISAC but I found some issue. below:

remark: I received certificate from FS-ISAC.

CLIENT CERTIFICATE: CERTIFICATE -> cert.pem & PRIVATE KEY -> cert.key

I am not sure that I upload file type correctly.

Could you recommend me?

Re: Prototype for FS-ISAC

lmori — Sun, 02 Jul 2017 08:27:10 GMT

Hi @iThreatHunt,

could you open the two files with a text editor and check the contents ?

You should see an header like this for the certificate (public key):

-----BEGIN CERTIFICATE-----
...

Did you upload the Server CA ?

Re: Prototype for FS-ISAC

iThreatHunt — Mon, 03 Jul 2017 05:30:02 GMT

Hi @lmori

taxii.py (from minemeld.core) : I upload follow this code (.pem & .crt)

Node Confiuguration : success

But this node cannot retrieve data from FS-ISAC. Whrere is application log for investigate?

Re: Prototype for FS-ISAC

lmori — Mon, 03 Jul 2017 07:06:38 GMT

Hi @iThreatHunt,

you can check the logs downloading minemeld-engine.log from System > Dashboard > Engine > Logs.

Could you check on the FS-ISAC WebUI what is the time of the last indicator in the feed you are polling ? By default the Miner the first time it polls the source it polls the last day worth of indicators, if you want to go further back in time you should configure the initial_interval parameter in the prototype:

initial_interval: 7d

Re: Prototype for FS-ISAC

iThreatHunt — Wed, 05 Jul 2017 02:06:00 GMT

Thanks @lmori. It is works.

Re: Prototype for FS-ISAC

xgil2017 — Thu, 03 Aug 2017 10:27:22 GMT

Hi @lmori

I have one similar problem synchronizing local minemeld with fsisac cyber repository.

...."collection fsisac not found" appears when minemeld try to polling isac.

Any idea?

Thanks.

Re: Prototype for FS-ISAC

lmori — Fri, 04 Aug 2017 10:32:46 GMT

Hi @xgil2017,

I think the collection name is wrong, it should be in the form <fs-isac username>.<feedname>

Luigi

Re: Prototype for FS-ISAC

Shankar_Chandrasekhar — Thu, 17 Aug 2017 14:40:58 GMT

Hi Lugi,

I am stuck here not sure what this error code means

Re: Prototype for FS-ISAC

npalviriya — Thu, 18 Jan 2018 09:53:14 GMT

Hi Luigi @lmori

I have follow your configuration and get all data with share_level=red.

BTW, my customer would like to have all shre_level=green data.

I have customize prototype as attached screenshot and commit failed.

Could you please suggest the correct configuration?

Thanks

Nattapon

Re: Prototype for FS-ISAC

hshawn — Fri, 06 Jul 2018 18:49:35 GMT

Where are you all even getting the feed names and discovery_service URL for the feed? I am looking all over the portal and I don't see anything with TAXII feed information anywhere in there.

the URL mentioned in this thread (analysis.fsisac.com...) does not seem to be a real thing. Anyone have a working FS-ISAC feed into Minemeld? Can you provide some details about how to find the required info for populating the variables here?

TIA

Re: Prototype for FS-ISAC

lukasj — Wed, 17 Oct 2018 16:51:32 GMT

I'm running minemeld 0.9.50 on the latest RHEL 7.5.

Now i tried to attach the the FS-ISAC feed, username feed, cert, seems to be fine.

When i manuall pull the feed from the Node; "LAST RUN" receives the State ERROR: 'module' object has no attrbute 'sslwrap'.

/opt/minemeld/log/minemeld-engine.log tells me:

2018-10-17T18:30:40 (12180)basepoller._polling_loop INFO: Polling fs-isac-soltra-feed
2018-10-17T18:30:41 (12180)basepoller._poll ERROR: Exception in polling loop for fs-isac-soltra-feed: 'module' object has no attribute 'sslwrap'

Traceback (most recent call last):
File "/opt/minemeld/engine/core/minemeld/ft/basepoller.py", line 721, in _poll
    performed = self._polling_loop()
File "/opt/minemeld/engine/core/minemeld/ft/basepoller.py", line 571, in _polling_loop
    iterator = self._build_iterator(now)
File "/opt/minemeld/engine/core/minemeld/ft/taxii.py", line 1131, in _build_iterator
    self._discover_services(tc)
File "/opt/minemeld/engine/core/minemeld/ft/taxii.py", line 292, in _discover_services
    resp = self._call_taxii_service(self.discovery_service, tc, request)
File "/opt/minemeld/engine/core/minemeld/ft/taxii.py", line 282, in _call_taxii_service
    port=port
File "/opt/minemeld/engine/current/lib/python2.7/site-packages/libtaxii/clients.py", line 337, in call_taxii_service2
    response = urllib.request.urlopen(req)
File "/usr/lib64/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
File "/usr/lib64/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
File "/usr/lib64/python2.7/urllib2.py", line 449, in _open
    '_open', req)
File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
File "/opt/minemeld/engine/current/lib/python2.7/site-packages/libtaxii/clients.py", line 363, in https_open
    return self.do_open(self.get_connection, req)
File "/usr/lib64/python2.7/urllib2.py", line 1211, in do_open
    h.request(req.get_method(), req.get_selector(), req.data, headers)
File "/usr/lib64/python2.7/httplib.py", line 1041, in request
    self._send_request(method, url, body, headers)
File "/usr/lib64/python2.7/httplib.py", line 1075, in _send_request
    self.endheaders(body)
File "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders
    self._send_output(message_body)
File "/usr/lib64/python2.7/httplib.py", line 881, in _send_output
    self.send(msg)
File "/usr/lib64/python2.7/httplib.py", line 843, in send
    self.connect()
File "/opt/minemeld/engine/current/lib/python2.7/site-packages/libtaxii/clients.py", line 443, in connect
    ca_certs=self.ca_certs)
File "/opt/minemeld/engine/current/lib/python2.7/site-packages/gevent/_ssl2.py", line 410, in wrap_socket
    ciphers=ciphers)
File "/opt/minemeld/engine/current/lib/python2.7/site-packages/gevent/_ssl2.py", line 84, in __init__
    self._sslobj = _ssl.sslwrap(self._sock, server_side,

AttributeError: 'module' object has no attribute 'sslwrap'

Maybe someone else ran into the same/similar issue and knows how to fix this ?

It seems to be related to the python code handling the certificate/ssl/tls connectivity.

Re: Prototype for FS-ISAC

benime — Wed, 24 Oct 2018 17:54:06 GMT

@hshawn wrote:

Where are you all even getting the feed names and discovery_service URL for the feed? I am looking all over the portal and I don't see anything with TAXII feed information anywhere in there.

the URL mentioned in this thread (analysis.fsisac.com...) does not seem to be a real thing. Anyone have a working FS-ISAC feed into Minemeld? Can you provide some details about how to find the required info for populating the variables here?

TIA

I don't know if you ever solved this, but if you don't have access to analysis.fsisac.com you'll need to request it from FS-ISAC support.

Once you have access, there's a Publish link at the top of the page that allows you to create your own custom feed based on the information you want (URLs, IPs, file hashes, etc.) and other criteria. You'll use the feed name combined with your credentials to access it.

HTH

Re: Prototype for FS-ISAC

lukasj — Fri, 26 Oct 2018 05:01:03 GMT

FYI, in case anyone runs into this issue i described earlier:

/opt/minemeld/log/minemeld-engine.log tells me:

2018-10-17T18:30:41 (12180)basepoller._poll ERROR: Exception in polling loop for <your miner node>: 'module' object has no attribute 'sslwrap'

This can be solved by replacing minemelds internal python "gevent" with a newer version.

For whatever reason, minmeld brings it own "gevent" in /opt/minemeld/engine/current/lib/python2.7/site-packages/gevent this outdated gevent version seems to cause issue with the Python Version installed on RHEL 7.

Just install the latest Version (pip install --upgrade gevent) and then replace the minemeld "gevent" with the new version from /usr/lib64/python2.7/site-packages/gevent.

After that the FS-ISAC Feed/Miner (and also other feeds requiring certificate authentication) is working fine on RHEL 7.

Re: Prototype for FS-ISAC

Carlos_Gomes — Wed, 10 Jul 2019 18:51:42 GMT

Has anyone successfully stripped http and https from the FSISAC Feed so that PaloAlto FWs can block those URLs? Currently aggregating the URLs into an output works and PAs can pull them into EDLs however it is pulling them with http and https which PAs then are not able to block those objects, according to PA article any url object cannot contain http:// or https://

I found the example here https://live.paloaltonetworks.com/t5/MineMeld-Articles/Using-MineMeld-to-Create-a-Custom-Miner/tac-p/275802#M327

It shows how to do it against a csv/txt list but i have not been able to find it for the taxi client feed. I applied the same terminology but that did not work. Anyone else has been able to do this and can share some insight?

Re: Prototype for FS-ISAC

hshawn — Mon, 15 Jul 2019 20:41:58 GMT

Thanks to all the pieces everyone has provided to this puzzle in this thread I was able to finally get a FS-ISAC feed setup however it errors out when polling.

I noticed in the log files I was seeing:

2019-07-15T13:35:07 (2425)basepoller._actor_loop INFO: FS-ISAC-Feed-1563221567853 - command: 1563222907645 poll
2019-07-15T13:35:07 (2425)basepoller._polling_loop INFO: Polling FS-ISAC-Feed-1563221567853
2019-07-15T13:35:07 (2425)basepoller._poll ERROR: Exception in polling loop for FS-ISAC-Feed-1563221567853: global name 'HTTPSClientAuthHandler' is not defined
Traceback (most recent call last):
  File "/opt/minemeld/engine/core/minemeld/ft/basepoller.py", line 724, in _poll
    performed = self._polling_loop()
  File "/opt/minemeld/engine/core/minemeld/ft/basepoller.py", line 571, in _polling_loop
    iterator = self._build_iterator(now)
  File "/opt/minemeld/engine/core/minemeld/ft/taxii.py", line 1131, in _build_iterator
    self._discover_services(tc)
  File "/opt/minemeld/engine/core/minemeld/ft/taxii.py", line 292, in _discover_services
    resp = self._call_taxii_service(self.discovery_service, tc, request)
  File "/opt/minemeld/engine/core/minemeld/ft/taxii.py", line 282, in _call_taxii_service
    port=port
  File "/opt/minemeld/engine/current/lib/python2.7/site-packages/libtaxii/clients.py", line 307, in call_taxii_service2
    handler_list.append(HTTPSClientAuthHandler(k, c))
NameError: global name 'HTTPSClientAuthHandler' is not defined
2019-07-15T13:35:11 (2425)basepoller._polling_loop INFO: Polling FS-ISAC-Feed-1563221567853
2019-07-15T13:35:11 (2425)basepoller._poll ERROR: Exception in polling loop for FS-ISAC-Feed-1563221567853: global name 'HTTPSClientAuthHandler' is not defined
Traceback (most recent call last):
  File "/opt/minemeld/engine/core/minemeld/ft/basepoller.py", line 724, in _poll
    performed = self._polling_loop()
  File "/opt/minemeld/engine/core/minemeld/ft/basepoller.py", line 571, in _polling_loop
    iterator = self._build_iterator(now)
  File "/opt/minemeld/engine/core/minemeld/ft/taxii.py", line 1131, in _build_iterator
    self._discover_services(tc)
  File "/opt/minemeld/engine/core/minemeld/ft/taxii.py", line 292, in _discover_services
    resp = self._call_taxii_service(self.discovery_service, tc, request)
  File "/opt/minemeld/engine/core/minemeld/ft/taxii.py", line 282, in _call_taxii_service
    port=port
  File "/opt/minemeld/engine/current/lib/python2.7/site-packages/libtaxii/clients.py", line 307, in call_taxii_service2
    handler_list.append(HTTPSClientAuthHandler(k, c))
NameError: global name 'HTTPSClientAuthHandler' is not defined
2019-07-15T13:35:12 (2425)basepoller._actor_loop INFO: FS-ISAC-Feed-1563221567853 - command: 1563222907645 age_out
2019-07-15T13:35:12 (2425)table._query_by_index INFO: Deleted in scan of _age_out: 0
2019-07-15T13:35:12 (2425)basepoller._actor_loop INFO: FS-ISAC-Feed-1563221567853 - command: 1563222907645 gc
2019-07-15T13:35:12 (2425)table._query_by_index INFO: Deleted in scan of _withdrawn: 0

I tried the change suggested by @lukasj but that just hard downed Minemeld until I replaced the gevent directory with the old one again.

This is the error I am running into, what am I missing? The cert and username/password fields have green checks

Any suggestions?

TIA!

Re: Prototype for FS-ISAC

Carlos_Gomes — Tue, 16 Jul 2019 18:59:06 GMT

Can you share what your prototype config looks like, here is mine. This is working for me without any problems.

Re: Prototype for FS-ISAC

hshawn — Tue, 16 Jul 2019 19:13:37 GMT

Sure thing, Here is our prototype:

Re: Prototype for FS-ISAC

Carlos_Gomes — Tue, 16 Jul 2019 19:18:15 GMT

So if i were i would setup exactly as i have as you do not need minemeld on the source and collection names.

TAGS:

ConfidenceHigh ShareLevelRed

CONFIG:

age_out:
default: last_seen+30d
sudden_death: false
attributes:
confidence: 30
share_level: red
client_cert_required: true
collection: username.FSISAC_FEED
discovery_service: https://analysis.fsisac.com/taxii-discovery-service
initial_interval: 90d
source_name: fs-isac.username.FSISAC_FEED