Install MineMeld behind the proxy server

[ust_lynt]

Having issue to install the MM behind the proxy, the IP address of MM is whitelisted for any proto/url in the proxy configure.

While the installation is started thru the command 

sudo apt-get install minemeld 

 I do see the requests from MM is comming to the proxy and the process of the install seems like goes well enought until at some points it fails with the error

minemeld@minemeld-virtual-machine:~$  sudo apt-get install minemeld
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  minemeld
0 upgraded, 1 newly installed, 0 to remove and 335 not upgraded.
Need to get 0 B/24.3 kB of archives.
After this operation, 69.6 kB of additional disk space will be used.
Selecting previously unselected package minemeld.
(Reading database ... 172862 files and directories currently installed.)
Preparing to unpack .../minemeld_0.9.11-2build1_amd64.deb ...
Unpacking minemeld (0.9.11-2build1) ...
Processing triggers for ureadahead (0.100.0-16) ...
Setting up minemeld (0.9.11-2build1) ...
adduser: The user `minemeld' already exists. Exiting.
2018-02-09 07:44:07,254 ERROR:0.9.11 Symbolic link current in /opt/minemeld/engine not found: [Errno 2] No such file or directory: '/opt/minemeld/engine/current'
2018-02-09 07:44:07,254 ERROR:0.9.11 Symbolic link current in /opt/minemeld/www not found: [Errno 2] No such file or directory: '/opt/minemeld/www/current'
2018-02-09 07:44:07,254 ERROR:0.9.11 Symbolic link current in /opt/minemeld/prototypes not found: [Errno 2] No such file or directory: '/opt/minemeld/prototypes/current'
2018-02-09 07:44:07,255 INFO:0.9.11 Current status:
2018-02-09 07:44:07,255 INFO:0.9.11 minemeld-engine: current: None latest: None
2018-02-09 07:44:07,255 INFO:0.9.11 minemeld-webui: current: None latest: None
2018-02-09 07:44:07,255 INFO:0.9.11 minemeld-prototypes: current: None latest: None
2018-02-09 07:49:05,957 CRITICAL:0.9.11 Error running curl: curl: (7) Failed to connect to minemeld-updates.panw.io port 80: Connection timed out

Traceback (most recent call last):
  File "/usr/sbin/minemeld-auto-update", line 787, in <module>
    main()
  File "/usr/sbin/minemeld-auto-update", line 742, in main
    update_package_list(config)
  File "/usr/sbin/minemeld-auto-update", line 443, in update_package_list
    config
  File "/usr/sbin/minemeld-auto-update", line 407, in download_package_list
    curl_options=config.get('curl-options', None)
  File "/usr/sbin/minemeld-auto-update", line 303, in download_via_curl
    stderr=subprocess.STDOUT
  File "/usr/lib/python2.7/subprocess.py", line 573, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command 'curl -k  -f -s -A "MineMeld/0.9.11" --show-error http://minemeld-updates.panw.io/stage2/0_9/manifest.json -o /tmp/mmaupackages0nlrvB' returned non-zero exit status 7
 * Restarting nginx nginx                                                                                                                                                                    [ OK ]
 * Restarting statistics collection and monitoring daemon collectd                                                                                                                           [ OK ]
rsyslog stop/waiting
rsyslog start/running, process 10999

While checking this erro log, I can see it tries to reach 

2018-02-09 07:49:05,957 CRITICAL:0.9.11 Error running curl: curl: (7) Failed to connect to minemeld-updates.panw.io port 80: Connection timed out

Command 'curl -k -f -s -A "MineMeld/0.9.11" --show-error http://minemeld-updates.panw.io/stage2/0_9/manifest.json -o /tmp/mmaupackages0nlrvB' returned non-zero exit status 7

and fails to reach this link on port 80, which is not make sense since while issue the command from minemeld user enverement it works great and catching the files

curl  minemeld-updates.panw.io:80
curl -k  -f -s -A "MineMeld/0.9.11" --show-error http://minemeld-updates.panw.io/stage2/0_9/manifest.json

The weard part is when I trace the logs in the Firewall, it seems like the curl makes the requests to reach minemeld-updates.panw.io:80 directly to the internet while installing the minemeld and ignoring the proxy settings

So, my quetions is why some of the requests would be directed to proxy while some of them are sent directly to firewall when the MM is installed  

  Here is my enverentment parameters

minemeld@minemeld-virtual-machine:~$ cat /etc/environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
http_proxy="http://x.x.x.x:1080/"
https_proxy="https://x.x.x.x:1080/"
ftp_proxy="ftp://x.x.x.x:1080/"
no_proxy="localhost,127.0.0.1,localaddress,.localdomain.com"
HTTP_PROXY="http://x.x.x.x:1080/"
HTTPS_PROXY="https://x.x.x.x:1080/"
FTP_PROXY="ftp://x.x.x.x:1080/"
NO_PROXY="localhost,127.0.0.1,localaddress,.localdomain.com"

minemeld@minemeld-virtual-machine:~$ cat /etc/apt/apt.conf.d/95proxies
Acquire::http::proxy "http://x.x.x.x:1080/";
Acquire::https::proxy "https://x.x.x.x:1080/";
Acquire::ftp::proxy "ftp://x.x.x.x:1080/";

minemeld@minemeld-virtual-machine:~$ cat /etc/default/minemeld
export HTTP_PROXY=http://x.x.x.x:1080
export HTTPS_PROXY=https://x.x.x.x:1080

[ust_lynt · Accepted solution]

Just resolved it, in case somebody would be interesting in..

I missed that 'sudo' does not preserve the environment  settings by default. Gooing 'root' to the sake of this installation had resolved the issue

Regards.

[ay.marusov]

Hello ust_lynt!

Could you please ellaborate on how did you solve this problem?

I've exported proxy settings to
/etc/apt/apt.conf
/etc/environment
/etc/profile
/etc/default/minemeld

and still no success. I've tried to install it using sudo -i command and directly from root - it didn't help. What did you mean by saying that going root solved your problem?

UPDATE:

After much struggle, creating configuration file .curlrc in user's home directory (root in my case) finally have resolved this issue. Correct syntax is this:

proxy = <proxy_host>:<proxy_port> 

More thoroghly explained here:

https://stackoverflow.com/questions/7559103/how-to-setup-curl-to-permanently-use-a-proxy