Nodes polling error

L0 Member

Nodes polling error

Hello somewho have an idea?

Installed Minemeld on an fresh Ubuntu 14.0.4 like the manual installation guide.

Import the Office365 configuration 

All Nodes got an SSL Error message see below

 

2017-04-19T12:45:54 (22890)basepoller.hup INFO: office365_O365 - hup received, force polling
2017-04-19T12:45:54 (22890)basepoller._huppable_wait INFO: hup is clear: False
2017-04-19T12:45:54 (22890)basepoller._actor_loop INFO: office365_O365 - command: 1492598754316 poll
2017-04-19T12:45:54 (22890)basepoller._polling_loop INFO: Polling office365_O365
2017-04-19T12:45:54 (22890)connectionpool._new_conn INFO: Starting new HTTPS connection (1): support.content.office.net
2017-04-19T12:45:54 (22890)basepoller._poll ERROR: Exception in polling loop for office365_O365: [Errno bad handshake] [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]
Traceback (most recent call last):
File "/opt/minemeld/engine/0.9.36.post2/local/lib/python2.7/site-packages/minemeld/ft/basepoller.py", line 701, in _poll
performed = self._polling_loop()
File "/opt/minemeld/engine/0.9.36.post2/local/lib/python2.7/site-packages/minemeld/ft/basepoller.py", line 568, in _polling_loop
iterator = self._build_iterator(now)
File "/opt/minemeld/engine/0.9.36.post2/local/lib/python2.7/site-packages/minemeld/ft/o365.py", line 165, in _build_iterator
oiterator = self._o365_iterator(now)
File "/opt/minemeld/engine/0.9.36.post2/local/lib/python2.7/site-packages/minemeld/ft/o365.py", line 115, in _o365_iterator
r = _session.send(prepreq, **rkwargs)
File "/opt/minemeld/engine/0.9.36.post2/local/lib/python2.7/site-packages/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/opt/minemeld/engine/0.9.36.post2/local/lib/python2.7/site-packages/requests/adapters.py", line 431, in send
raise SSLError(e, request=request)
SSLError: [Errno bad handshake] [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]

 

Any guidance that can be provided would be greatly appreciated! 

 

Thanks Holger

L7 Applicator

Re: Nodes polling error

Hi @HolgerKiene,

certificate verification is failing. Are you behind a proxy or a device doing SSL decryption ?

Could you open a shell on the MineMeld instance, issue the following and report back any error you see ?

$ cd /tmp/ && wget https://support.content.office.net/en-us/static/O365IPAddresses.xml

Thanks,

luigi

L0 Member

Re: Nodes polling error

Thanks for for your fast answer,

You're right my mistake

 

Holger

 

L7 Applicator

Re: Nodes polling error

Hi @HolgerKiene,

thanks for taking the time to tell us everything is working !

 

luigi

L2 Linker

Re: Nodes polling error

Another node get error:

 

dcadmin@MICMM01:/tmp$ wget https://check.torproject.org/exit-addresses

--2017-10-19 12:54:22--  https://check.torproject.org/exit-addresses

Resolving check.torproject.org (check.torproject.org)... 146.112.61.106, ::ffff:146.112.61.106

Connecting to check.torproject.org (check.torproject.org)|146.112.61.106|:443... connected.

ERROR: cannot verify check.torproject.org's certificate, issued by ‘/CN=Cisco Umbrella Secondary SubCA dfw-SG/O=Cisco’:

  Unable to locally verify the issuer's authority.

To connect to check.torproject.org insecurely, use `--no-check-certificate'.

dcadmin@MICMM01:/tmp$

 

Can I change the prototype to request http rather than https?

Tor Exit Node:

https://check.torproject.org/exit-addresses

L5 Sessionator

Re: Nodes polling error

@clockhart : are you aware of the hailataxii.guest_blutmagie_de_torExits prototype in the standard library that also "mines" the tor exit nodes? Any reason not to use it?

 

I've just realized you're receiving a certificate error from Cisco Umbrella. That means that your MineMeld instance is using a secure proxy to reach the feed (SSL man-in-the-middle). In such a case you need to import the related certificates in the MineMeld's trust ring.

L2 Linker

Re: Nodes polling error

Good point but my Office365 https requests work behind same DNS proxy. I believe customer is using OpenDNS so that makes sense. I'll take a look at the other prototype to see if I get the same error. I appreciate the response.

L2 Linker

Re: Nodes polling error

Set up my miner, aggregator and output nodes but no luck. hailataxi Miner reports 273 indicators, which is considerably lower than the tor-exit.nodes (913). Is there a reason for the discrepancy?

 

Also am I using the wrong aggregator? My list is empty.

L7 Applicator

Re: Nodes polling error

Hi @clockhart,

to track tor nodes please use blutmagie.* prototypes, I have found them more reliable over time.

One reason you could considerably less nodes from hailataxii is caused by how TAXII DataFeed work. TAXII DataFeeds are designed to publish updates, not full current lists of indicators. This means that the 273 nodes you see are most probably the 273 tor nodes most recently added to the list of active tor nodes, not the full list. Blutmagie.* and tor.* prototypes instead provide the full current list of Tor nodes.

 

Hope this helps.

 

luigi

L2 Linker

Re: Nodes polling error

Luigi,

 

Thanks for the response. I'm using blutmagie now for my miner and it's looking good. Appreciate the assistance.

 

cpl

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!