Minemeld behind corporate proxy

[calamari]

Following all the requirements for proxy configuration - https://live.paloaltonetworks.com/t5/MineMeld-Discussions/Minemeld-with-Proxy/m-p/77356/highlight/tr...

I can perform a curl request, for example curl https://www.spamhaus.org/drop/edrop.txt and get back a response.

/etc/default/minemeld is configured with the proxy settings as well as /etc/environment.

But still getting an error:

2017-02-20T00:04:30 (212)basepoller._poll ERROR: Exception in polling loop for spamhaus_DROP: HTTPSConnectionPool(host='www.spamhaus.org', port=443): Max retries exceeded with url: /drop/drop.txt (Caused by ProxyError('Cannot connect to proxy.', gaierror(-5, 'No address associated with hostname')))
Traceback (most recent call last):
File "/opt/minemeld/engine/0.9.30/local/lib/python2.7/site-packages/minemeld/ft/basepoller.py", line 468, in _poll
performed = self._polling_loop()
File "/opt/minemeld/engine/0.9.30/local/lib/python2.7/site-packages/minemeld/ft/basepoller.py", line 352, in _polling_loop
iterator = self._build_iterator(now)
File "/opt/minemeld/engine/0.9.30/local/lib/python2.7/site-packages/minemeld/ft/http.py", line 191, in _build_iterator
**rkwargs
File "/opt/minemeld/engine/0.9.30/local/lib/python2.7/site-packages/requests/api.py", line 69, in get
return request('get', url, params=params, **kwargs)
File "/opt/minemeld/engine/0.9.30/local/lib/python2.7/site-packages/requests/api.py", line 50, in request
response = session.request(method=method, url=url, **kwargs)
File "/opt/minemeld/engine/0.9.30/local/lib/python2.7/site-packages/requests/sessions.py", line 465, in request
resp = self.send(prep, **send_kwargs)
File "/opt/minemeld/engine/0.9.30/local/lib/python2.7/site-packages/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/opt/minemeld/engine/0.9.30/local/lib/python2.7/site-packages/requests/adapters.py", line 424, in send
raise ConnectionError(e, request=request)

Any other configuration required?

[lmori]

Hi @calamari,

from the error (Cannot connect to proxy.', gaierror(-5, 'No address associated with hostname')))) it seems that the MineMeld instance cannot resolved the proxy DNS Name.

Could you issue the following command on the MM instance ?

$ dig <proxy hostname>

[calamari]

Thanks, but the proxy is setup as an IP address rather than a hostname. Not sure why MM would be looking for a hostname. perhaps I could just define one in /etc/hosts.

[lmori]

Could you double check the HTTP_PROXY and HTTPS_PROXY variables ? They should look like:

export HTTP_PROXY="http://10.10.1.10:3128"
export HTTPS_PROXY="http://10.10.1.10:1080"

[calamari]

Within /etc/default/minemeld I have the proxy settings set as you said. The only difference is that the proxy requires authentication, so I have the entries in the form:

export HTTP_PROXY="http://domain%5Cusername:password@10.10.1.10:3128"
export HTTPS_PROXY="http://domain%5Cusername:password@10.10.1.10:1080"

[lmori]

Hi @calamari,

please could you try removing the domain from the variables and see if the error in the log becomes an authentication error ?

export HTTP_PROXY="http://username:password@10.10.1.10:3128"
export HTTPS_PROXY="http://username:password@10.10.1.10:1080"