New minemeld deploy unable to login to GUI

drewdown · ‎04-20-2018

Used OVA to deploy it on ESXi. Default admin/minemeld did not work after deployment and NO changes. Gives the message "ERROR CHECKING CREDENTIALS: Bad Gateway"

Logged in via CLI and ran the following:

ubuntu@minemeld:~$ sudo htpasswd /opt/minemeld/local/config/api/wsgi.htpasswd admin
New password:
Re-type new password:
Updating password for user admin

ubuntu@minemeld:~$ sudo -u minemeld /opt/minemeld/engine/current/bin/supervisorctl -c /opt/minemeld/supervisor/config/supervisord.conf restart minemeld-web
minemeld-web: stopped
minemeld-web: ERROR (abnormal termination)

Still cant login after changing the password.

drewdown · ‎04-20-2018

Found fix below.

sudo service minemeld stop
sudo -H -u minemeld /opt/minemeld/engine/current/bin/pip install pip==9.0.3
sudo service minemeld start

View solution in original post

drewdown · ‎04-20-2018

Found fix below.

sudo service minemeld stop
sudo -H -u minemeld /opt/minemeld/engine/current/bin/pip install pip==9.0.3
sudo service minemeld start

View solution in original post

Rievax · ‎04-20-2018

Hi drewdown,

Thanks for the answer to the problem! I was looking for that since 3 days ago... apparently this is a new bug introduced lately:

https://github.com/PaloAltoNetworks/minemeld-core/issues/270

I just couldn't figure out how to revert to pip < v10

Cheers!

rsukhnanan · ‎04-23-2018

Thanks! this worked for me!

fwmike · ‎07-23-2018

Is there a permanent fix for this issue?

My MM instance keeps stopping and downgrading PIP per the instructions above revives it.

ubuntu@minemeld:~$ sudo service minemeld stop
* Stopping: minemeld [ OK ]
ubuntu@minemeld:~$ sudo -H -u minemeld /opt/minemeld/engine/current/bin/pip install pip==9.0.3
The directory '/home/minemeld/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/minemeld/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: pip==9.0.3 in /opt/minemeld/engine/0.9.48.post1/lib/python2.7/site-packages
/opt/minemeld/engine/0.9.48.post1/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py:339: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
SNIMissingWarning
/opt/minemeld/engine/0.9.48.post1/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
/opt/minemeld/engine/0.9.48.post1/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
You are using pip version 9.0.3, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
ubuntu@minemeld:~$ sudo service minemeld start
* Starting: minemeld Unlinking stale socket /var/run/minemeld/minemeld.sock
[ OK ]
ubuntu@minemeld:~$

lmori · ‎07-27-2018

@fwmike, fix has been merged long time ago. I guess your engine is stopping for some other reason. Please could you check your minemeld-engine.log file?

fwmike · ‎08-02-2018

Thanks for checking in. The failure happened again and the log shows low memory errors:

ubuntu@minemeld:~$ sudo grep memory  /opt/minemeld/log/minemeld-engine.log | tail -10
2018-08-02T13:57:21 (31689)amqp._blocked ERROR: Connection blocked: low on memory
2018-08-02T13:57:24 (31689)amqp._blocked ERROR: Connection blocked: low on memory
2018-08-02T13:57:24 (31689)amqp._blocked ERROR: Connection blocked: low on memory
2018-08-02T13:57:24 (31689)amqp._blocked ERROR: Connection blocked: low on memory
2018-08-02T13:57:24 (31689)amqp._blocked ERROR: Connection blocked: low on memory
2018-08-02T13:57:25 (31689)amqp._blocked ERROR: Connection blocked: low on memory
2018-08-02T13:57:25 (31689)amqp._blocked ERROR: Connection blocked: low on memory
2018-08-02T13:57:25 (31689)amqp._blocked ERROR: Connection blocked: low on memory
2018-08-02T13:57:25 (31689)amqp._blocked ERROR: Connection blocked: low on memory
2018-08-02T13:58:27 (31689)amqp._blocked ERROR: Connection blocked: low on memory
ubuntu@minemeld:~$

I restarted MM and see total Mem usage steadily increasing:

top - 14:51:54 up 55 days, 23:46,  2 users,  load average: 0.00, 0.01, 0.05
Tasks:  94 total,   2 running,  92 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:   1017736 total,   762240 used,   255496 free,    21184 buffers
KiB Swap:        0 total,        0 used,        0 free.   161496 cached Mem

  PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND                                                                                                                 
 7330 ubuntu    20   0  103080   2048   1060 S  0.3  0.2   0:02.18 sshd                                                                                                                    
 7399 minemeld  20   0   60608  14800   1896 S  0.3  1.5   0:00.77 supervisord                                                                                                             
 7419 minemeld  20   0  417048  92920   7464 S  0.3  9.1   0:07.02 gunicorn                                                                                                                
    1 root      20   0   33504   1416     56 S  0.0  0.1   0:01.53 init                                                                                                                    
    2 root      20   0       0      0      0 S  0.0  0.0   0:00.19 kthreadd                                                                                                                
    3 root      20   0       0      0      0 S  0.0  0.0   0:18.47 ksoftirqd/0                                                                                                             
    5 root       0 -20       0      0      0 S  0.0  0.0   0:00.00 kworker/0:0H                                                                                                            
    7 root      20   0       0      0      0 S  0.0  0.0   2:37.20 rcu_sched                                                                                                               
    8 root      20   0       0      0      0 R  0.0  0.0   4:03.51 rcuos/0                                                                                                                 
    9 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcu_bh                                                                                                                  
   10 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcuob/0                                                                                                                 
   11 root      rt   0       0      0      0 S  0.0  0.0   0:00.00 migration/0                                                                                                             
   12 root      rt   0       0      0      0 S  0.0  0.0   0:20.51 watchdog/0

VM has 1 CPU and 1 GB RAM. Will request additional RAM to see if it helps.

fwmike · ‎08-10-2018

Adding another gig of RAM appears to have resolved my issue:

ubuntu@minemeld:~$ free 
             total       used       free     shared    buffers     cached
Mem:       2049924    1950076      99848        672     206312     953164
-/+ buffers/cache:     790600    1259324
Swap:            0          0          0

FWIW, we have 13 miners, 8 procs and 8 outputs.

lmori · ‎08-10-2018

Hi @fwmike,

may I know which miners are you using and how many indicators are you processing?

Thanks,

luigi

fwmike · ‎08-13-2018

Strata

Prisma

Cortex

New minemeld deploy unable to login to GUI

New minemeld deploy unable to login to GUI

Show your appreciation!