Minemeld-web FATAL (Ubuntu 18.04)

LassiK — Thu, 11 Mar 2021 13:34:41 GMT

Ive been having a sweaty couple of days trying to get this to work and I must say ubuntu is not my forte.

Anyway this all started from Ubuntu update. We had 14.04 which had sudo vulnerability and I was tasked to update this OS.

Only thing running on it was this Minemeld and ofcourse after update it broke.

I think Ive finally got partly working, but subject is still broken. Ive been trying this forum and internet to fix without progress so far.

If I try to login from webpage it gives me error [ERROR CHECKING CREDENTIALS: BAD GATEWAY]

I check services:

sudo -u minemeld /opt/minemeld/engine/current/bin/supervisorctl -c /opt/minemeld/supervisor/config/supervisord.conf statusminemeld-engine RUNNING pid 3511, uptime 0:12:51
minemeld-supervisord-listener RUNNING pid 3510, uptime 0:12:51
minemeld-traced RUNNING pid 3512, uptime 0:12:51
minemeld-web FATAL Exited too quickly (process log may have details)

So this here is the problem right? There was even a time when I was able to get this minemeld-web RUNNING after I did minemeld update, but then I lost webpage to 404 error....

1. anyway here is the minemeld-web log as I have no idea how to read this:

less /opt/minemeld/log/minemeld-web.log
[2018-11-22 11:53:07 +0000] [12237] [INFO] Starting gunicorn 19.5.0
[2018-11-22 11:53:07 +0000] [12237] [INFO] Listening at: http://127.0.0.1:5000 (12237)
[2018-11-22 11:53:07 +0000] [12237] [INFO] Using worker: gevent
[2018-11-22 11:53:07 +0000] [12271] [INFO] Booting worker with pid: 12271
[2018-11-22 11:53:07 UTC] [12271] [INFO] MMLogger started
/opt/minemeld/engine/0.9.50.post4/local/lib/python2.7/site-packages/minemeld/flask/aaa.py:21: ExtDeprecationWarning: Importing flask.ext.login is deprecated, use flask_login instead.
import flask.ext.login
/opt/minemeld/engine/0.9.50.post4/local/lib/python2.7/site-packages/cryptography/hazmat/primitives/constant_time.py:26: CryptographyDeprecationWarning: Support for your Python version is deprecated. The next version of cryptography will remove support. Please upgrade to a 2.7.x release that supports hmac.compare_digest as soon as possible.
utils.DeprecatedIn23,
[2018-11-22 11:53:08 UTC] [12271] [INFO] Config loaded: {'MINEMELD_LOCAL_LIBRARY_PATH': '/opt/minemeld/local/library', 'USERS_DB': 'wsgi.htpasswd', 'MINEMELD_LOCAL_PROTOTYPE_PATH': '/opt/minemeld/local/prototypes', 'SUPERVISOR_URL': 'unix:///var/run/minemeld/minemeld.sock', 'MINEMELD_PROTOTYPE_PATH': '/opt/minemeld/local/prototypes/:/opt/minemeld/prototypes/current', 'MINEMELD_PIP_PATH': '/opt/minemeld/engine/current/bin/pip', 'MINEMELD_TRACED_PURGE_PATH': '/opt/minemeld/engine/current/bin/mm-traced-purge', 'API_AUTH_ENABLED': True, 'MINEMELD_LOG_DIRECTORY_PATH': '/opt/minemeld/log', 'MINEMELD_7Z_PATH': '/usr/bin/7z', 'MINEMELD_RESTORE_PATH': '/opt/minemeld/engine/current/bin/mm-restore', 'MINEMELD_LOCAL_PATH': '/opt/minemeld/local', 'MINEMELD_GIT_PATH': '/usr/bin/git', 'SNS_ENABLED': True}
[2018-11-22 11:53:08 UTC] [12271] [INFO] USERS_DB loaded from /opt/minemeld/local/config/api/wsgi.htpasswd
[2018-11-22 11:53:08 UTC] [12271] [INFO] FEEDS_USERS_DB loaded from /opt/minemeld/local/config/api/feeds.htpasswd
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] New uuid file created.
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] Instance uuid = 9548c961933c4e9f9298e99cd6be7a65
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] MineMeld cloud notification service is ready.
[2018-11-22 11:53:09 UTC] [12271] [INFO] Extensions frigidaire not found, paths not loaded
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/supervisor/minemeld-engine/restart' (HEAD, OPTIONS, GET) -> supervisor.restart_minemeld_engine>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/supervisor/minemeld-engine/start' (HEAD, OPTIONS, GET) -> supervisor.start_minemeld_engine>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/supervisor/minemeld-engine/stop' (HEAD, OPTIONS, GET) -> supervisor.stop_minemeld_engine>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/supervisor/minemeld-web/hup' (HEAD, OPTIONS, GET) -> supervisor.hup_minemeld_web>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/status/events/status' (HEAD, OPTIONS, GET) -> status.get_status_events>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/status/backup/import' (POST, OPTIONS) -> status.import_local_backup>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/aaa/users/current' (HEAD, OPTIONS, GET) -> aaa.get_current_user>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/extensions/git-install' (POST, OPTIONS) -> extensions.install_from_git>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/extensions/git-refs' (HEAD, OPTIONS, GET) -> extensions.get_git_refs>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/validate/syslogminerrule' (POST, OPTIONS) -> validate.validate_syslogminerrule>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/metrics/minemeld' (HEAD, OPTIONS, GET) -> metrics.get_global_metrics>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/config/committed' (HEAD, OPTIONS, GET) -> config.get_committed_config>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/traced/purge-all' (HEAD, OPTIONS, GET) -> traced.traced_purge_all>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/status/minemeld' (HEAD, OPTIONS, GET) -> status.get_minemeld_status>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/config/running' (HEAD, OPTIONS, GET) -> config.get_running_config>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/config/mgmtbus' (HEAD, OPTIONS, GET) -> config.get_mgmtbus>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/status/system' (HEAD, OPTIONS, GET) -> status.get_system_status>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/status/config' (HEAD, OPTIONS, GET) -> status.get_minemeld_running_config>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/status/backup' (POST, OPTIONS) -> status.generate_local_backup>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/status/mkwish' (POST, OPTIONS) -> status.sns_wish>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/config/reload' (HEAD, OPTIONS, GET) -> config.reload_running_config>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/config/commit' (POST, OPTIONS) -> config.commit>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/config/fabric' (HEAD, OPTIONS, GET) -> config.get_fabric>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/traced/query' (HEAD, OPTIONS, GET) -> traced.traced_query>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/status/info' (HEAD, OPTIONS, GET) -> status.get_system_info>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/config/info' (HEAD, OPTIONS, GET) -> config.get_config_info>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/config/full' (HEAD, OPTIONS, GET) -> config.get_config_full>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/config/node' (POST, OPTIONS) -> config.create_node>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/logs/minemeld-engine.log' (HEAD, OPTIONS, GET) -> logs.get_minemeld_engine_log>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/logs/minemeld-web.log' (HEAD, OPTIONS, GET) -> logs.get_minemeld_web_log>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/aaa/feeds' (HEAD, OPTIONS, GET) -> aaa.get_feeds>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/aaa/tags' (HEAD, OPTIONS, GET) -> aaa.get_tags>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/taxii-collection-management-service' (POST, OPTIONS) -> taxiicollmgmt.taxii_collection_mgmt_service>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/taxii-discovery-service' (POST, OPTIONS) -> taxiidiscovery.taxii_discovery_service>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/taxii-poll-service' (POST, OPTIONS) -> taxiipoll.taxii_poll_service>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/supervisor' (HEAD, OPTIONS, GET) -> supervisor.service_status>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/extensions' (HEAD, OPTIONS, GET) -> extensions.list_extensions>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/extensions' (POST, OPTIONS) -> extensions.upload_extension>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/prototype' (HEAD, OPTIONS, GET) -> prototype.list_prototypes>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/metrics/' (HEAD, OPTIONS, GET) -> metrics.get_metrics>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/logout' (HEAD, OPTIONS, GET) -> login.logout>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/login' (HEAD, POST, OPTIONS, GET) -> login.login>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/status/backup/import/<backup_id>/restore' (POST, OPTIONS) -> status.restore_local_backup>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/aaa/users/<subsystem>/<username>/attributes' (POST, OPTIONS) -> aaa.set_user_attributes>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/status/events/query/<quuid>' (HEAD, OPTIONS, GET) -> status.get_query_events>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/traced/query/<query_uuid>/kill' (HEAD, OPTIONS, GET) -> traced.traced_kill_query>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/config/data/<datafilename>/append' (POST, OPTIONS) -> configdata.append_config_data>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/status/<nodename>/signal/<signalname>' (HEAD, POST, OPTIONS, GET) -> status.signal_node>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/jobs/<job_group>/<jobid>/log' (HEAD, OPTIONS, GET) -> jobs.get_job_log>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/aaa/feeds/<feedname>/attributes' (PUT, POST, OPTIONS) -> aaa.set_feed_attributes>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/aaa/users/<subsystem>/<username>' (PUT, OPTIONS) -> aaa.set_user_password>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/aaa/users/<subsystem>/<username>' (OPTIONS, DELETE) -> aaa.delete_user>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/extensions/<extension>/deactivate' (HEAD, POST, OPTIONS, GET) -> extensions.deactivate_extension>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/extensions/<extension>/uninstall' (HEAD, POST, OPTIONS, GET) -> extensions.uninstall_extension>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/extensions/<extension>/activate' (POST, OPTIONS) -> extensions.activate_extension>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/metrics/minemeld/<nodetype>' (HEAD, OPTIONS, GET) -> metrics.get_node_type_metrics>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/metrics/<node>/<metric>' (HEAD, OPTIONS, GET) -> metrics.get_metric>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/status/backup/<jobid>' (HEAD, OPTIONS, GET) -> status.get_local_backup>
[2018-11-22 11:53:09 UTC] [12271] [DEBUG] app rule: <Rule '/config/node/<nodenum>' (HEAD, OPTIONS, GET) -> config.get_node>

2. Should this path have a config/ folder also?

/opt/minemeld/
engine/ local/ log/ prototypes/ supervisor/ www/

Anything else?

Re: Minemeld-web FATAL (Ubuntu 18.04)

LassiK — Thu, 11 Mar 2021 14:01:42 GMT

UPDATE:

Ofcourse it starts working right after you ask help... well "working"

After following this thread and doing these python RDDtool stuff, every service is running and I can access webpage.

Although something is still broken because when I do external dynamic list -> test URL it says access error.... more work, but its getting close

Re: Minemeld-web FATAL (Ubuntu 18.04)

lmori — Thu, 11 Mar 2021 14:03:46 GMT

Hi @LassiK ,

MineMeld binaries are not compatible with Ubuntu 18.04.

My suggestion for running it on 18.04 is using Docker image. Moving forward that will be the preferred way of deployment.

Luigi

topic Re: Minemeld-web FATAL (Ubuntu 18.04) in General Topics

Minemeld-web FATAL (Ubuntu 18.04)

Re: Minemeld-web FATAL (Ubuntu 18.04)

Re: Minemeld-web FATAL (Ubuntu 18.04)