Nodes polling error

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Please sign in to see details of an important advisory in our Customer Advisories area.

Nodes polling error

L0 Member

Hello somewho have an idea?

Installed Minemeld on an fresh Ubuntu 14.0.4 like the manual installation guide.

Import the Office365 configuration 

All Nodes got an SSL Error message see below

 

2017-04-19T12:45:54 (22890)basepoller.hup INFO: office365_O365 - hup received, force polling
2017-04-19T12:45:54 (22890)basepoller._huppable_wait INFO: hup is clear: False
2017-04-19T12:45:54 (22890)basepoller._actor_loop INFO: office365_O365 - command: 1492598754316 poll
2017-04-19T12:45:54 (22890)basepoller._polling_loop INFO: Polling office365_O365
2017-04-19T12:45:54 (22890)connectionpool._new_conn INFO: Starting new HTTPS connection (1): support.content.office.net
2017-04-19T12:45:54 (22890)basepoller._poll ERROR: Exception in polling loop for office365_O365: [Errno bad handshake] [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]
Traceback (most recent call last):
File "/opt/minemeld/engine/0.9.36.post2/local/lib/python2.7/site-packages/minemeld/ft/basepoller.py", line 701, in _poll
performed = self._polling_loop()
File "/opt/minemeld/engine/0.9.36.post2/local/lib/python2.7/site-packages/minemeld/ft/basepoller.py", line 568, in _polling_loop
iterator = self._build_iterator(now)
File "/opt/minemeld/engine/0.9.36.post2/local/lib/python2.7/site-packages/minemeld/ft/o365.py", line 165, in _build_iterator
oiterator = self._o365_iterator(now)
File "/opt/minemeld/engine/0.9.36.post2/local/lib/python2.7/site-packages/minemeld/ft/o365.py", line 115, in _o365_iterator
r = _session.send(prepreq, **rkwargs)
File "/opt/minemeld/engine/0.9.36.post2/local/lib/python2.7/site-packages/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/opt/minemeld/engine/0.9.36.post2/local/lib/python2.7/site-packages/requests/adapters.py", line 431, in send
raise SSLError(e, request=request)
SSLError: [Errno bad handshake] [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]

 

Any guidance that can be provided would be greatly appreciated! 

 

Thanks Holger

11 REPLIES 11

L7 Applicator

Hi @HolgerKiene,

certificate verification is failing. Are you behind a proxy or a device doing SSL decryption ?

Could you open a shell on the MineMeld instance, issue the following and report back any error you see ?

$ cd /tmp/ && wget https://support.content.office.net/en-us/static/O365IPAddresses.xml

Thanks,

luigi

Thanks for for your fast answer,

You're right my mistake

 

Holger

 

Hi @HolgerKiene,

thanks for taking the time to tell us everything is working !

 

luigi

Another node get error:

 

dcadmin@MICMM01:/tmp$ wget https://check.torproject.org/exit-addresses

--2017-10-19 12:54:22--  https://check.torproject.org/exit-addresses

Resolving check.torproject.org (check.torproject.org)... 146.112.61.106, ::ffff:146.112.61.106

Connecting to check.torproject.org (check.torproject.org)|146.112.61.106|:443... connected.

ERROR: cannot verify check.torproject.org's certificate, issued by ‘/CN=Cisco Umbrella Secondary SubCA dfw-SG/O=Cisco’:

  Unable to locally verify the issuer's authority.

To connect to check.torproject.org insecurely, use `--no-check-certificate'.

dcadmin@MICMM01:/tmp$

 

Can I change the prototype to request http rather than https?

Tor Exit Node:

https://check.torproject.org/exit-addresses

@clockhart : are you aware of the hailataxii.guest_blutmagie_de_torExits prototype in the standard library that also "mines" the tor exit nodes? Any reason not to use it?

 

I've just realized you're receiving a certificate error from Cisco Umbrella. That means that your MineMeld instance is using a secure proxy to reach the feed (SSL man-in-the-middle). In such a case you need to import the related certificates in the MineMeld's trust ring.

Good point but my Office365 https requests work behind same DNS proxy. I believe customer is using OpenDNS so that makes sense. I'll take a look at the other prototype to see if I get the same error. I appreciate the response.

Set up my miner, aggregator and output nodes but no luck. hailataxi Miner reports 273 indicators, which is considerably lower than the tor-exit.nodes (913). Is there a reason for the discrepancy?

 

Also am I using the wrong aggregator? My list is empty.

Hi @clockhart,

to track tor nodes please use blutmagie.* prototypes, I have found them more reliable over time.

One reason you could considerably less nodes from hailataxii is caused by how TAXII DataFeed work. TAXII DataFeeds are designed to publish updates, not full current lists of indicators. This means that the 273 nodes you see are most probably the 273 tor nodes most recently added to the list of active tor nodes, not the full list. Blutmagie.* and tor.* prototypes instead provide the full current list of Tor nodes.

 

Hope this helps.

 

luigi

Luigi,

 

Thanks for the response. I'm using blutmagie now for my miner and it's looking good. Appreciate the assistance.

 

cpl

L0 Member

Hello, I'm just getting started with MineMeld. We have an internal block IP and URL feeds that are hosted on a web server, a text file hosted via HTTPS page. My issue is that this server does not have a valid certificate, but It's my internal server, so I don't care. Is there a way to ignore certificate errors when pulling HTTPS feeds? 

@jniedenthal : That behavior of the HttpFT class is controlled by the verify_cert boolean configuration attibute (defaults to true). Add the attribute to your prototype with the value set to "False"

  • 14370 Views
  • 11 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!