- Access exclusive content
- Connect with peers
- Share your expertise
- Find support resources
Enhanced Security Measures in Place: To ensure a safer experience, we’ve implemented additional, temporary security measures for all users.
on 03-12-2019 06:45 AM - edited on 12-14-2021 06:01 AM by jforsythe
Note: Palo Alto Networks made an end-of-life announcement about the MineMeld™ application in AutoFocus™ on August 1, 2021. Please read this article to learn about our recommended migration options.
An easy way to use MineMeld is installing the binary packages on an Ubuntu 16.04 instance.
Note: Binary packages are only available for 64 bits architectures.
Ubuntu Server LTS 16.04 (64 bits)
First thing you should harden your new instance. MineMeld won't take of this for you. A good tutorial is this one.
Update all the packages of the instance before installing.
sudo apt update && sudo apt dist-upgrade -y
You can use the following commands to configure iptables to allow sessions on ports used by MineMeld. Also these rules drop all IPv6 traffic, if you are running MineMeld in an IPv6 network make sure you change the suggested rules.
sudo apt install -y iptables-persistent
sudo iptables -A INPUT -i lo -j ACCEPT
sudo iptables -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
sudo iptables -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
sudo iptables -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
sudo iptables -A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
sudo iptables -A INPUT -p tcp -m tcp --dport 13514 -j ACCEPT
sudo iptables -A INPUT -p icmp -m icmp --icmp-type 0 -j ACCEPT
sudo iptables -A INPUT -p icmp -m icmp --icmp-type 3 -j ACCEPT
sudo iptables -A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT
sudo iptables -A INPUT -p icmp -m icmp --icmp-type 11 -j ACCEPT
sudo iptables -P INPUT DROP
sudo iptables -P FORWARD DROP
sudo bash -c "iptables-save > /etc/iptables/rules.v4"
sudo ip6tables -A INPUT -i lo -j ACCEPT
sudo ip6tables -P INPUT DROP
sudo ip6tables -P FORWARD DROP
sudo bash -c "ip6tables-save > /etc/iptables/rules.v6"
Add the MineMeld repo GPG key to the APT trusted keyring:
wget -qO - https://minemeld-updates.panw.io/gpg.key | sudo apt-key add -
Double check the GPG key fingerprint, to make sure it is matching the official MineMeld GPG key (fingerprint should match characters in bold):
apt-key adv --fingerprint DD0DA1F9
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --homedir /tmp/tmp.W74MaAG3pI --no-auto-check-trustdb --trust-model always --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --fingerprint DD0DA1F9
pub 4096R/DD0DA1F9 2016-07-15
Key fingerprint = E558 CE6E 3968 0F31 8F6C BFAC B401 E02E DD0D A1F9
uid Palo Alto Networks, MineMeld Team <minemeld@paloaltonetworks.com>
[...]
Add the MineMeld APT repo to the system list and update the apt cache:
sudo add-apt-repository "deb http://minemeld-updates.panw.io/ubuntu xenial-minemeld main"
sudo apt update
MineMeld requires nginx and redis. We have to install them before installing MineMeld package to avoid configuration conflicts:
sudo apt install -y nginx redis-server
Install the MineMeld infrastructure package via apt. This will also automatically trigger the download of the latest MineMeld packages.
sudo apt install -o Dpkg::Options::="--force-overwrite" -y minemeld
We should restart the instance to make sure all the configurations are applied and all the services are started in the right order:
sudo shutdown -r now
Check if the 3 MineMeld services are up and running:
$ sudo -u minemeld /opt/minemeld/engine/current/bin/supervisorctl -c /opt/minemeld/local/supervisor/config/supervisord.conf status
minemeld-engine RUNNING pid 3727, uptime 0:08:50
minemeld-traced RUNNING pid 3728, uptime 0:08:50
minemeld-web RUNNING pid 3729, uptime 0:08:50
minemeld-supervisord-listener RUNNING pid 3730, uptime 0:08:50
Done! Check the Quick Tour article to get started.
Hello,
I am getting these errors after installing minemeld. System returned no error during install.
Any thoughts would be highly appriciated.
Thanks,
administrator@ubuntu:~$ sudo -u minemeld /opt/minemeld/engine/current/bin/supervisorctl -c /opt/minemeld/supervisor/config/supervisord.conf status
[sudo] password for administrator:
sudo: /opt/minemeld/engine/current/bin/supervisorctl: command not found
administrator@ubuntu:~$ ps -ef | grep mine
adminis+ 937 921 0 14:58 pts/0 00:00:00 grep --color=auto mine
administrator@ubuntu:~$ systemctl start minemeld
Failed to start minemeld.service: The name org.freedesktop.PolicyKit1 was not provided by any .service files
See system logs and 'systemctl status minemeld.service' for details.
administrator@ubuntu:~$ systemctl status minemeld.service
● minemeld.service - Process Monitoring and Control Daemon
Loaded: loaded (/lib/systemd/system/minemeld.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Tue 2019-03-12 14:57:54 PDT; 1min 32s ago
Process: 882 ExecStart=/opt/minemeld/engine/current/bin/supervisord -c /opt/minemeld/supervisor/config/supervisord.conf --pidfile /var/run/minemeld/minemeld.pid (code=exited, status=203/EXEC)
Process: 876 ExecStartPre=/bin/chown -R minemeld:minemeld /var/run/minemeld/ (code=exited, status=0/SUCCESS)
Process: 863 ExecStartPre=/bin/mkdir /var/run/minemeld (code=exited, status=0/SUCCESS)
Mar 12 14:57:54 ubuntu systemd[1]: Starting Process Monitoring and Control Daemon...
Mar 12 14:57:54 ubuntu systemd[1]: minemeld.service: Control process exited, code=exited status=203
Mar 12 14:57:54 ubuntu systemd[1]: Failed to start Process Monitoring and Control Daemon.
Mar 12 14:57:54 ubuntu systemd[1]: minemeld.service: Unit entered failed state.
Mar 12 14:57:54 ubuntu systemd[1]: minemeld.service: Failed with result 'exit-code'.
is ubuntu 19.04 supported?
getting the following error
PLAY [minemeld playbook] ********************************************************************************************************************************************************************************************************************
TASK [Gathering Facts] **********************************************************************************************************************************************************************************************************************
ok: [127.0.0.1]
TASK [infrastructure : debug] ***************************************************************************************************************************************************************************************************************
ok: [127.0.0.1] => {
"msg": "Loading vars for Ubuntu 19.04"
}
TASK [infrastructure : include_vars] ********************************************************************************************************************************************************************************************************
fatal: [127.0.0.1]: FAILED! => {"msg": "No file was found when using with_first_found. Use the 'skip: true' option to allow this task to be skipped if no files are found"}
to retry, use: --limit @/home/brian/minemeld-ansible/local.retry
PLAY RECAP **********************************************************************************************************************************************************************************************************************************
127.0.0.1 : ok=2 changed=0 unreachable=0 failed=1
Step 1 Issues:
If you see the following just after you entered “sudo apt update && sudo apt dist-upgrade -y”:
….
Err:2 cdrom://Ubuntu-Server 16.04.6 LTS _Xenial Xerus_ - Release amd64 (20190226) xenial Release
Please use apt-cdrom to make this CD-ROM recognized by APT. apt-get update cannot be used to add new CD-ROMs
….
Do the following:
and comment or delte the following line
deb cdrom:[Ubuntu-Server 16.04 LTS _Xenial Xerus_ - Release amd64 (20160420.3)]/ xenial main restricted
(NOTE - where it says 20160420.3 it might be a different number for you, so don’t get hung up on it!)
Step 5 issues:
If you get “sudo: add-apt-repository: command not found”, type the following:
FYI - I'm running MM on VMware 10.1.5 and using the Ubuntu-16.04.6-server-amd64.iso
Tip
I installed open-ssh before gonig through the steps for my VM to make entering commands easyer - to do this enter the following:
I installed Minemeld on a fresh Ubuntu 16.04.6 LTS with the instructions above but the minemeld-engine is hanging in STARTING state. minemeld-engine.log shows a couple of directory not found messages for CheckPoints. Any idea how to fix this?
lab-user@minemeld:~$ sudo -u minemeld /opt/minemeld/engine/current/bin/supervisorctl -c /opt/minemeld/local/supervisor/config/supervisord.conf status [sudo] password for lab-user: minemeld-engine STARTING minemeld-supervisord-listener RUNNING pid 1190, uptime 0:00:19 minemeld-traced RUNNING pid 1192, uptime 0:00:19 minemeld-web RUNNING pid 1193, uptime 0:00:19
2019-07-06T18:21:20 (4767)launcher.main INFO: Starting mm-run.py version 0.9.62 2019-07-06T18:21:20 (4767)launcher.main INFO: mm-run.py arguments: Namespace(config='/opt/minemeld/local/config', multiprocessing=0, nodes_per_chassis=15.0, verbose=False) 2019-07-06T18:21:21 (4767)config._load_config_from_dir INFO: Switching to candidate config 2019-07-06T18:21:21 (4767)config._load_config_from_dir INFO: Changes in config: [_ConfigChange(nodename='spamhaus_EDROP', nodeclass='minemeld.ft.http.HttpFT', change=0, detail=None), _ConfigChange(nodename='dshield_blocklist', nodeclass='minemeld.ft.http.HttpFT', change=0, detail=None), _ConfigChange(nodename='inboundaggregator', nodeclass='minemeld.ft.ipop.AggregateIPv4FT', change=0, detail=None), _ConfigChange(nodename='inboundfeedhc', nodeclass='minemeld.ft.redis.RedisSet', change=0, detail=None), _ConfigChange(nodename='spamhaus_DROP', nodeclass='minemeld.ft.http.HttpFT', change=0, detail=None), _ConfigChange(nodename='wlWhiteListIPv4', nodeclass='minemeld.ft.local.YamlIPv4FT', change=0, detail=None), _ConfigChange(nodename='inboundfeedlc', nodeclass='minemeld.ft.redis.RedisSet', change=0, detail=None), _ConfigChange(nodename='inboundfeedmc', nodeclass='minemeld.ft.redis.RedisSet', change=0, detail=None)] 2019-07-06T18:21:21 (4767)config._destroy_old_nodes INFO: Destroyed nodes: [] 2019-07-06T18:21:21 (4767)launcher.main INFO: mm-run.py config: _Config(nodes={'spamhaus_EDROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.EDROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/edrop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'dshield_blocklist': {'output': True, 'config': {'indicator': {'regex': '^([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})\\t([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})', 'transform': '\\1-\\2'}, 'source_name': 'dshield.block', 'age_out': {'default': None, 'sudden_death': True, 'interval': 257}, 'url': 'https://www.dshield.org/block.txt', 'fields': {'dshield_name': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t([^\\t]+)', 'transform': '\\1'}, 'dshield_country': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t([A-Z]+)', 'transform': '\\1'}, 'dshield_nattacks': {'regex': '^.*\\t.*\\t[0-9]+\\t([0-9]+)', 'transform': '\\1'}, 'dshield_email': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t[A-Z]+\\t(\\S+)', 'transform': '\\1'}}, 'interval': 619, 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '[#S].*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'inboundaggregator': {'inputs': ['spamhaus_DROP', 'spamhaus_EDROP', 'dshield_blocklist', 'wlWhiteListIPv4'], 'config': {'whitelist_prefixes': ['wl'], 'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", "direction == 'inbound'"], 'name': 'accept inbound IPv4', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", 'direction == null'], 'name': 'accept generic IPv4', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.ipop.AggregateIPv4FT', 'output': True}, 'inboundfeedhc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence > 75', "share_level == 'green'"], 'name': 'accept confidence > 75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'spamhaus_DROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.DROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/drop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'wlWhiteListIPv4': {'inputs': [], 'config': {'attributes': {'confidence': 100, 'share_level': 'red'}, 'interval': 3600, 'age_out': {'default': None, 'sudden_death': True, 'interval': 67}}, 'class': 'minemeld.ft.local.YamlIPv4FT', 'output': True}, 'inboundfeedlc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence < 50', "share_level == 'green'"], 'name': 'accept confidence < 50 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'inboundfeedmc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence >= 50', 'confidence < 75', "share_level == 'green'"], 'name': 'accept confidence 50-75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}}, fabric={'config': {'priority': -2, 'num_connections': 50}, 'class': 'ZMQRedis'}, mgmtbus={'slave': {}, 'master': {}, 'transport': {'config': {'priority': 2, 'num_connections': 10}, 'class': 'ZMQRedis'}}, changes=[_ConfigChange(nodename='spamhaus_EDROP', nodeclass='minemeld.ft.http.HttpFT', change=0, detail=None), _ConfigChange(nodename='dshield_blocklist', nodeclass='minemeld.ft.http.HttpFT', change=0, detail=None), _ConfigChange(nodename='inboundaggregator', nodeclass='minemeld.ft.ipop.AggregateIPv4FT', change=0, detail=None), _ConfigChange(nodename='inboundfeedhc', nodeclass='minemeld.ft.redis.RedisSet', change=0, detail=None), _ConfigChange(nodename='spamhaus_DROP', nodeclass='minemeld.ft.http.HttpFT', change=0, detail=None), _ConfigChange(nodename='wlWhiteListIPv4', nodeclass='minemeld.ft.local.YamlIPv4FT', change=0, detail=None), _ConfigChange(nodename='inboundfeedlc', nodeclass='minemeld.ft.redis.RedisSet', change=0, detail=None), _ConfigChange(nodename='inboundfeedmc', nodeclass='minemeld.ft.redis.RedisSet', change=0, detail=None)]) 2019-07-06T18:21:21 (4767)launcher.main INFO: multiprocessing: #cores: 2 2019-07-06T18:21:21 (4767)launcher.main INFO: multiprocessing: max #chassis: 2 2019-07-06T18:21:21 (4767)launcher.main INFO: Number of chassis: 1 Traceback (most recent call last): File "/opt/minemeld/engine/current/bin/mm-run", line 10, in <module> sys.exit(main()) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 252, in main minemeld.comm.cleanup(config.fabric['class'], config.fabric['config']) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/__init__.py", line 14, in cleanup return ZMQRedis.cleanup(config) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/zmqredis.py", line 855, in cleanup tkeys = SR.keys(pattern='mm:topic:*') File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 936, in keys return self.execute_command('KEYS', pattern) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 578, in execute_command connection.send_command(*args) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 563, in send_command self.send_packed_command(self.pack_command(*args)) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 538, in send_packed_command self.connect() File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 442, in connect raise ConnectionError(self._error_message(e)) redis.exceptions.ConnectionError: Error 2 connecting to unix socket: /var/run/redis/redis.sock. No such file or directory. 2019-07-06T18:21:33 (4953)launcher.main INFO: Starting mm-run.py version 0.9.62 2019-07-06T18:21:33 (4953)launcher.main INFO: mm-run.py arguments: Namespace(config='/opt/minemeld/local/config', multiprocessing=0, nodes_per_chassis=15.0, verbose=False) 2019-07-06T18:21:35 (4953)config._load_config_from_dir INFO: Switching to candidate config 2019-07-06T18:21:35 (4953)config._load_config_from_dir INFO: Changes in config: [] 2019-07-06T18:21:35 (4953)config._destroy_old_nodes INFO: Destroyed nodes: [] 2019-07-06T18:21:35 (4953)launcher.main INFO: mm-run.py config: _Config(nodes={'spamhaus_EDROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.EDROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/edrop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'dshield_blocklist': {'output': True, 'config': {'indicator': {'regex': '^([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})\\t([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})', 'transform': '\\1-\\2'}, 'source_name': 'dshield.block', 'age_out': {'default': None, 'sudden_death': True, 'interval': 257}, 'url': 'https://www.dshield.org/block.txt', 'fields': {'dshield_name': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t([^\\t]+)', 'transform': '\\1'}, 'dshield_country': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t([A-Z]+)', 'transform': '\\1'}, 'dshield_nattacks': {'regex': '^.*\\t.*\\t[0-9]+\\t([0-9]+)', 'transform': '\\1'}, 'dshield_email': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t[A-Z]+\\t(\\S+)', 'transform': '\\1'}}, 'interval': 619, 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '[#S].*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'inboundaggregator': {'inputs': ['spamhaus_DROP', 'spamhaus_EDROP', 'dshield_blocklist', 'wlWhiteListIPv4'], 'config': {'whitelist_prefixes': ['wl'], 'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", "direction == 'inbound'"], 'name': 'accept inbound IPv4', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", 'direction == null'], 'name': 'accept generic IPv4', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.ipop.AggregateIPv4FT', 'output': True}, 'inboundfeedhc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence > 75', "share_level == 'green'"], 'name': 'accept confidence > 75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'spamhaus_DROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.DROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/drop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'wlWhiteListIPv4': {'inputs': [], 'config': {'attributes': {'confidence': 100, 'share_level': 'red'}, 'interval': 3600, 'age_out': {'default': None, 'sudden_death': True, 'interval': 67}}, 'class': 'minemeld.ft.local.YamlIPv4FT', 'output': True}, 'inboundfeedlc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence < 50', "share_level == 'green'"], 'name': 'accept confidence < 50 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'inboundfeedmc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence >= 50', 'confidence < 75', "share_level == 'green'"], 'name': 'accept confidence 50-75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}}, fabric={'config': {'priority': -2, 'num_connections': 50}, 'class': 'ZMQRedis'}, mgmtbus={'slave': {}, 'master': {}, 'transport': {'config': {'priority': 2, 'num_connections': 10}, 'class': 'ZMQRedis'}}, changes=[]) 2019-07-06T18:21:35 (4953)launcher.main INFO: multiprocessing: #cores: 2 2019-07-06T18:21:35 (4953)launcher.main INFO: multiprocessing: max #chassis: 2 2019-07-06T18:21:35 (4953)launcher.main INFO: Number of chassis: 1 Traceback (most recent call last): File "/opt/minemeld/engine/current/bin/mm-run", line 10, in <module> sys.exit(main()) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 252, in main minemeld.comm.cleanup(config.fabric['class'], config.fabric['config']) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/__init__.py", line 14, in cleanup return ZMQRedis.cleanup(config) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/zmqredis.py", line 855, in cleanup tkeys = SR.keys(pattern='mm:topic:*') File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 936, in keys return self.execute_command('KEYS', pattern) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 578, in execute_command connection.send_command(*args) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 563, in send_command self.send_packed_command(self.pack_command(*args)) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 538, in send_packed_command self.connect() File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 442, in connect raise ConnectionError(self._error_message(e)) redis.exceptions.ConnectionError: Error 2 connecting to unix socket: /var/run/redis/redis.sock. No such file or directory. 2019-07-06T18:21:36 (4969)launcher.main INFO: Starting mm-run.py version 0.9.62 2019-07-06T18:21:36 (4969)launcher.main INFO: mm-run.py arguments: Namespace(config='/opt/minemeld/local/config', multiprocessing=0, nodes_per_chassis=15.0, verbose=False) 2019-07-06T18:21:38 (4969)config._load_config_from_dir INFO: Switching to candidate config 2019-07-06T18:21:38 (4969)config._load_config_from_dir INFO: Changes in config: [] 2019-07-06T18:21:38 (4969)config._destroy_old_nodes INFO: Destroyed nodes: [] 2019-07-06T18:21:38 (4969)launcher.main INFO: mm-run.py config: _Config(nodes={'spamhaus_EDROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.EDROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/edrop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'dshield_blocklist': {'output': True, 'config': {'indicator': {'regex': '^([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})\\t([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})', 'transform': '\\1-\\2'}, 'source_name': 'dshield.block', 'age_out': {'default': None, 'sudden_death': True, 'interval': 257}, 'url': 'https://www.dshield.org/block.txt', 'fields': {'dshield_name': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t([^\\t]+)', 'transform': '\\1'}, 'dshield_country': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t([A-Z]+)', 'transform': '\\1'}, 'dshield_nattacks': {'regex': '^.*\\t.*\\t[0-9]+\\t([0-9]+)', 'transform': '\\1'}, 'dshield_email': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t[A-Z]+\\t(\\S+)', 'transform': '\\1'}}, 'interval': 619, 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '[#S].*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'inboundaggregator': {'inputs': ['spamhaus_DROP', 'spamhaus_EDROP', 'dshield_blocklist', 'wlWhiteListIPv4'], 'config': {'whitelist_prefixes': ['wl'], 'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", "direction == 'inbound'"], 'name': 'accept inbound IPv4', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", 'direction == null'], 'name': 'accept generic IPv4', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.ipop.AggregateIPv4FT', 'output': True}, 'inboundfeedhc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence > 75', "share_level == 'green'"], 'name': 'accept confidence > 75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'spamhaus_DROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.DROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/drop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'wlWhiteListIPv4': {'inputs': [], 'config': {'attributes': {'confidence': 100, 'share_level': 'red'}, 'interval': 3600, 'age_out': {'default': None, 'sudden_death': True, 'interval': 67}}, 'class': 'minemeld.ft.local.YamlIPv4FT', 'output': True}, 'inboundfeedlc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence < 50', "share_level == 'green'"], 'name': 'accept confidence < 50 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'inboundfeedmc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence >= 50', 'confidence < 75', "share_level == 'green'"], 'name': 'accept confidence 50-75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}}, fabric={'config': {'priority': -2, 'num_connections': 50}, 'class': 'ZMQRedis'}, mgmtbus={'slave': {}, 'master': {}, 'transport': {'config': {'priority': 2, 'num_connections': 10}, 'class': 'ZMQRedis'}}, changes=[]) 2019-07-06T18:21:38 (4969)launcher.main INFO: multiprocessing: #cores: 2 2019-07-06T18:21:38 (4969)launcher.main INFO: multiprocessing: max #chassis: 2 2019-07-06T18:21:38 (4969)launcher.main INFO: Number of chassis: 1 Traceback (most recent call last): File "/opt/minemeld/engine/current/bin/mm-run", line 10, in <module> sys.exit(main()) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 252, in main minemeld.comm.cleanup(config.fabric['class'], config.fabric['config']) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/__init__.py", line 14, in cleanup return ZMQRedis.cleanup(config) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/zmqredis.py", line 855, in cleanup tkeys = SR.keys(pattern='mm:topic:*') File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 936, in keys return self.execute_command('KEYS', pattern) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 578, in execute_command connection.send_command(*args) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 563, in send_command self.send_packed_command(self.pack_command(*args)) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 538, in send_packed_command self.connect() File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 442, in connect raise ConnectionError(self._error_message(e)) redis.exceptions.ConnectionError: Error 2 connecting to unix socket: /var/run/redis/redis.sock. No such file or directory. 2019-07-06T18:21:40 (4974)launcher.main INFO: Starting mm-run.py version 0.9.62 2019-07-06T18:21:40 (4974)launcher.main INFO: mm-run.py arguments: Namespace(config='/opt/minemeld/local/config', multiprocessing=0, nodes_per_chassis=15.0, verbose=False) 2019-07-06T18:21:42 (4974)config._load_config_from_dir INFO: Switching to candidate config 2019-07-06T18:21:42 (4974)config._load_config_from_dir INFO: Changes in config: [] 2019-07-06T18:21:42 (4974)config._destroy_old_nodes INFO: Destroyed nodes: [] 2019-07-06T18:21:42 (4974)launcher.main INFO: mm-run.py config: _Config(nodes={'spamhaus_EDROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.EDROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/edrop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'dshield_blocklist': {'output': True, 'config': {'indicator': {'regex': '^([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})\\t([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})', 'transform': '\\1-\\2'}, 'source_name': 'dshield.block', 'age_out': {'default': None, 'sudden_death': True, 'interval': 257}, 'url': 'https://www.dshield.org/block.txt', 'fields': {'dshield_name': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t([^\\t]+)', 'transform': '\\1'}, 'dshield_country': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t([A-Z]+)', 'transform': '\\1'}, 'dshield_nattacks': {'regex': '^.*\\t.*\\t[0-9]+\\t([0-9]+)', 'transform': '\\1'}, 'dshield_email': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t[A-Z]+\\t(\\S+)', 'transform': '\\1'}}, 'interval': 619, 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '[#S].*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'inboundaggregator': {'inputs': ['spamhaus_DROP', 'spamhaus_EDROP', 'dshield_blocklist', 'wlWhiteListIPv4'], 'config': {'whitelist_prefixes': ['wl'], 'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", "direction == 'inbound'"], 'name': 'accept inbound IPv4', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", 'direction == null'], 'name': 'accept generic IPv4', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.ipop.AggregateIPv4FT', 'output': True}, 'inboundfeedhc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence > 75', "share_level == 'green'"], 'name': 'accept confidence > 75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'spamhaus_DROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.DROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/drop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'wlWhiteListIPv4': {'inputs': [], 'config': {'attributes': {'confidence': 100, 'share_level': 'red'}, 'interval': 3600, 'age_out': {'default': None, 'sudden_death': True, 'interval': 67}}, 'class': 'minemeld.ft.local.YamlIPv4FT', 'output': True}, 'inboundfeedlc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence < 50', "share_level == 'green'"], 'name': 'accept confidence < 50 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'inboundfeedmc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence >= 50', 'confidence < 75', "share_level == 'green'"], 'name': 'accept confidence 50-75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}}, fabric={'config': {'priority': -2, 'num_connections': 50}, 'class': 'ZMQRedis'}, mgmtbus={'slave': {}, 'master': {}, 'transport': {'config': {'priority': 2, 'num_connections': 10}, 'class': 'ZMQRedis'}}, changes=[]) 2019-07-06T18:21:42 (4974)launcher.main INFO: multiprocessing: #cores: 2 2019-07-06T18:21:42 (4974)launcher.main INFO: multiprocessing: max #chassis: 2 2019-07-06T18:21:42 (4974)launcher.main INFO: Number of chassis: 1 Traceback (most recent call last): File "/opt/minemeld/engine/current/bin/mm-run", line 10, in <module> sys.exit(main()) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 252, in main minemeld.comm.cleanup(config.fabric['class'], config.fabric['config']) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/__init__.py", line 14, in cleanup return ZMQRedis.cleanup(config) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/zmqredis.py", line 855, in cleanup tkeys = SR.keys(pattern='mm:topic:*') File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 936, in keys return self.execute_command('KEYS', pattern) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 578, in execute_command connection.send_command(*args) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 563, in send_command self.send_packed_command(self.pack_command(*args)) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 538, in send_packed_command self.connect() File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 442, in connect raise ConnectionError(self._error_message(e)) redis.exceptions.ConnectionError: Error 2 connecting to unix socket: /var/run/redis/redis.sock. No such file or directory. 2019-07-06T18:21:46 (4979)launcher.main INFO: Starting mm-run.py version 0.9.62 2019-07-06T18:21:46 (4979)launcher.main INFO: mm-run.py arguments: Namespace(config='/opt/minemeld/local/config', multiprocessing=0, nodes_per_chassis=15.0, verbose=False) 2019-07-06T18:21:47 (4979)config._load_config_from_dir INFO: Switching to candidate config 2019-07-06T18:21:47 (4979)config._load_config_from_dir INFO: Changes in config: [] 2019-07-06T18:21:47 (4979)config._destroy_old_nodes INFO: Destroyed nodes: [] 2019-07-06T18:21:47 (4979)launcher.main INFO: mm-run.py config: _Config(nodes={'spamhaus_EDROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.EDROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/edrop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'dshield_blocklist': {'output': True, 'config': {'indicator': {'regex': '^([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})\\t([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})', 'transform': '\\1-\\2'}, 'source_name': 'dshield.block', 'age_out': {'default': None, 'sudden_death': True, 'interval': 257}, 'url': 'https://www.dshield.org/block.txt', 'fields': {'dshield_name': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t([^\\t]+)', 'transform': '\\1'}, 'dshield_country': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t([A-Z]+)', 'transform': '\\1'}, 'dshield_nattacks': {'regex': '^.*\\t.*\\t[0-9]+\\t([0-9]+)', 'transform': '\\1'}, 'dshield_email': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t[A-Z]+\\t(\\S+)', 'transform': '\\1'}}, 'interval': 619, 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '[#S].*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'inboundaggregator': {'inputs': ['spamhaus_DROP', 'spamhaus_EDROP', 'dshield_blocklist', 'wlWhiteListIPv4'], 'config': {'whitelist_prefixes': ['wl'], 'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", "direction == 'inbound'"], 'name': 'accept inbound IPv4', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", 'direction == null'], 'name': 'accept generic IPv4', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.ipop.AggregateIPv4FT', 'output': True}, 'inboundfeedhc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence > 75', "share_level == 'green'"], 'name': 'accept confidence > 75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'spamhaus_DROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.DROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/drop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'wlWhiteListIPv4': {'inputs': [], 'config': {'attributes': {'confidence': 100, 'share_level': 'red'}, 'interval': 3600, 'age_out': {'default': None, 'sudden_death': True, 'interval': 67}}, 'class': 'minemeld.ft.local.YamlIPv4FT', 'output': True}, 'inboundfeedlc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence < 50', "share_level == 'green'"], 'name': 'accept confidence < 50 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'inboundfeedmc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence >= 50', 'confidence < 75', "share_level == 'green'"], 'name': 'accept confidence 50-75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}}, fabric={'config': {'priority': -2, 'num_connections': 50}, 'class': 'ZMQRedis'}, mgmtbus={'slave': {}, 'master': {}, 'transport': {'config': {'priority': 2, 'num_connections': 10}, 'class': 'ZMQRedis'}}, changes=[]) 2019-07-06T18:21:47 (4979)launcher.main INFO: multiprocessing: #cores: 2 2019-07-06T18:21:47 (4979)launcher.main INFO: multiprocessing: max #chassis: 2 2019-07-06T18:21:47 (4979)launcher.main INFO: Number of chassis: 1 Traceback (most recent call last): File "/opt/minemeld/engine/current/bin/mm-run", line 10, in <module> sys.exit(main()) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 252, in main minemeld.comm.cleanup(config.fabric['class'], config.fabric['config']) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/__init__.py", line 14, in cleanup return ZMQRedis.cleanup(config) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/zmqredis.py", line 855, in cleanup tkeys = SR.keys(pattern='mm:topic:*') File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 936, in keys return self.execute_command('KEYS', pattern) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 578, in execute_command connection.send_command(*args) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 563, in send_command self.send_packed_command(self.pack_command(*args)) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 538, in send_packed_command self.connect() File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 442, in connect raise ConnectionError(self._error_message(e)) redis.exceptions.ConnectionError: Error 2 connecting to unix socket: /var/run/redis/redis.sock. No such file or directory. 2019-07-06T20:22:28 (1191)launcher.main INFO: Starting mm-run.py version 0.9.62 2019-07-06T20:22:28 (1191)launcher.main INFO: mm-run.py arguments: Namespace(config='/opt/minemeld/local/config', multiprocessing=0, nodes_per_chassis=15.0, verbose=False) 2019-07-06T20:22:30 (1191)config._load_config_from_dir INFO: Switching to candidate config 2019-07-06T20:22:30 (1191)config._load_config_from_dir INFO: Changes in config: [] 2019-07-06T20:22:30 (1191)config._destroy_old_nodes INFO: Destroyed nodes: [] 2019-07-06T20:22:30 (1191)launcher.main INFO: mm-run.py config: _Config(nodes={'spamhaus_EDROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.EDROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/edrop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'dshield_blocklist': {'output': True, 'config': {'indicator': {'regex': '^([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})\\t([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})', 'transform': '\\1-\\2'}, 'source_name': 'dshield.block', 'age_out': {'default': None, 'sudden_death': True, 'interval': 257}, 'url': 'https://www.dshield.org/block.txt', 'fields': {'dshield_name': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t([^\\t]+)', 'transform': '\\1'}, 'dshield_country': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t([A-Z]+)', 'transform': '\\1'}, 'dshield_nattacks': {'regex': '^.*\\t.*\\t[0-9]+\\t([0-9]+)', 'transform': '\\1'}, 'dshield_email': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t[A-Z]+\\t(\\S+)', 'transform': '\\1'}}, 'interval': 619, 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '[#S].*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'inboundaggregator': {'inputs': ['spamhaus_DROP', 'spamhaus_EDROP', 'dshield_blocklist', 'wlWhiteListIPv4'], 'config': {'whitelist_prefixes': ['wl'], 'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", "direction == 'inbound'"], 'name': 'accept inbound IPv4', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", 'direction == null'], 'name': 'accept generic IPv4', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.ipop.AggregateIPv4FT', 'output': True}, 'inboundfeedhc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence > 75', "share_level == 'green'"], 'name': 'accept confidence > 75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'spamhaus_DROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.DROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/drop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'wlWhiteListIPv4': {'inputs': [], 'config': {'attributes': {'confidence': 100, 'share_level': 'red'}, 'interval': 3600, 'age_out': {'default': None, 'sudden_death': True, 'interval': 67}}, 'class': 'minemeld.ft.local.YamlIPv4FT', 'output': True}, 'inboundfeedlc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence < 50', "share_level == 'green'"], 'name': 'accept confidence < 50 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'inboundfeedmc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence >= 50', 'confidence < 75', "share_level == 'green'"], 'name': 'accept confidence 50-75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}}, fabric={'config': {'priority': -2, 'num_connections': 50}, 'class': 'ZMQRedis'}, mgmtbus={'slave': {}, 'master': {}, 'transport': {'config': {'priority': 2, 'num_connections': 10}, 'class': 'ZMQRedis'}}, changes=[]) 2019-07-06T20:22:30 (1191)launcher.main INFO: multiprocessing: #cores: 2 2019-07-06T20:22:30 (1191)launcher.main INFO: multiprocessing: max #chassis: 2 2019-07-06T20:22:30 (1191)launcher.main INFO: Number of chassis: 1 2019-07-06T20:22:30 (1203)loader.load INFO: Loading minemeld_nodes:minemeld.ft.http.HttpFT 2019-07-06T20:22:30 (1203)base.read_checkpoint ERROR: spamhaus_EDROP - Error reading last checkpoint Traceback (most recent call last): File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/ft/base.py", line 255, in read_checkpoint with open(self.name+'.chkp', 'r') as f: IOError: [Errno 2] No such file or directory: 'spamhaus_EDROP.chkp' 2019-07-06T20:22:30 (1203)base.state INFO: spamhaus_EDROP - transitioning to state 1 2019-07-06T20:22:30 (1203)loader.load INFO: Loading minemeld_nodes:minemeld.ft.http.HttpFT 2019-07-06T20:22:30 (1203)base.read_checkpoint ERROR: dshield_blocklist - Error reading last checkpoint Traceback (most recent call last): File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/ft/base.py", line 255, in read_checkpoint with open(self.name+'.chkp', 'r') as f: IOError: [Errno 2] No such file or directory: 'dshield_blocklist.chkp' 2019-07-06T20:22:30 (1203)base.state INFO: dshield_blocklist - transitioning to state 1 2019-07-06T20:22:30 (1203)loader.load INFO: Loading minemeld_nodes:minemeld.ft.redis.RedisSet 2019-07-06T20:22:30 (1203)redis.read_checkpoint ERROR: inboundfeedlc - Error reading last checkpoint Traceback (most recent call last): File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/ft/redis.py", line 69, in read_checkpoint raise ValueError('{} - last checkpoint not found'.format(self.name)) ValueError: inboundfeedlc - last checkpoint not found 2019-07-06T20:22:30 (1203)base.connect INFO: inboundfeedlc - requesting fabric sub channel for inboundaggregator 2019-07-06T20:22:30 (1203)base.state INFO: inboundfeedlc - transitioning to state 1 2019-07-06T20:22:30 (1203)loader.load INFO: Loading minemeld_nodes:minemeld.ft.redis.RedisSet 2019-07-06T20:22:30 (1203)redis.read_checkpoint ERROR: inboundfeedhc - Error reading last checkpoint Traceback (most recent call last): File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/ft/redis.py", line 69, in read_checkpoint raise ValueError('{} - last checkpoint not found'.format(self.name)) ValueError: inboundfeedhc - last checkpoint not found 2019-07-06T20:22:30 (1203)base.connect INFO: inboundfeedhc - requesting fabric sub channel for inboundaggregator 2019-07-06T20:22:30 (1203)base.state INFO: inboundfeedhc - transitioning to state 1 2019-07-06T20:22:30 (1203)loader.load INFO: Loading minemeld_nodes:minemeld.ft.http.HttpFT 2019-07-06T20:22:30 (1203)base.read_checkpoint ERROR: spamhaus_DROP - Error reading last checkpoint Traceback (most recent call last): File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/ft/base.py", line 255, in read_checkpoint with open(self.name+'.chkp', 'r') as f: IOError: [Errno 2] No such file or directory: 'spamhaus_DROP.chkp' 2019-07-06T20:22:30 (1203)base.state INFO: spamhaus_DROP - transitioning to state 1 2019-07-06T20:22:30 (1203)loader.load INFO: Loading minemeld_nodes:minemeld.ft.local.YamlIPv4FT 2019-07-06T20:22:30 (1203)base.read_checkpoint ERROR: wlWhiteListIPv4 - Error reading last checkpoint Traceback (most recent call last): File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/ft/base.py", line 255, in read_checkpoint with open(self.name+'.chkp', 'r') as f: IOError: [Errno 2] No such file or directory: 'wlWhiteListIPv4.chkp' 2019-07-06T20:22:30 (1203)base.state INFO: wlWhiteListIPv4 - transitioning to state 1 2019-07-06T20:22:30 (1203)loader.load INFO: Loading minemeld_nodes:minemeld.ft.ipop.AggregateIPv4FT 2019-07-06T20:22:30 (1203)base.read_checkpoint ERROR: inboundaggregator - Error reading last checkpoint Traceback (most recent call last): File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/ft/base.py", line 255, in read_checkpoint with open(self.name+'.chkp', 'r') as f: IOError: [Errno 2] No such file or directory: 'inboundaggregator.chkp' 2019-07-06T20:22:30 (1203)base.connect INFO: inboundaggregator - requesting fabric sub channel for spamhaus_DROP 2019-07-06T20:22:30 (1203)base.connect INFO: inboundaggregator - requesting fabric sub channel for spamhaus_EDROP 2019-07-06T20:22:30 (1203)base.connect INFO: inboundaggregator - requesting fabric sub channel for dshield_blocklist 2019-07-06T20:22:30 (1203)base.connect INFO: inboundaggregator - requesting fabric sub channel for wlWhiteListIPv4 2019-07-06T20:22:30 (1203)base.state INFO: inboundaggregator - transitioning to state 1 2019-07-06T20:22:30 (1203)loader.load INFO: Loading minemeld_nodes:minemeld.ft.redis.RedisSet 2019-07-06T20:22:30 (1203)redis.read_checkpoint ERROR: inboundfeedmc - Error reading last checkpoint Traceback (most recent call last): File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/ft/redis.py", line 69, in read_checkpoint raise ValueError('{} - last checkpoint not found'.format(self.name)) ValueError: inboundfeedmc - last checkpoint not found 2019-07-06T20:22:30 (1203)base.connect INFO: inboundfeedmc - requesting fabric sub channel for inboundaggregator 2019-07-06T20:22:30 (1203)base.state INFO: inboundfeedmc - transitioning to state 1 2019-07-06T20:22:30 (1191)mgmtbus.init_graph INFO: state: {u'mbus:slave:wlWhiteListIPv4': {u'checkpoint': None, u'is_source': True, u'state': 1}, u'mbus:slave:spamhaus_DROP': {u'checkpoint': None, u'is_source': True, u'state': 1}, u'mbus:slave:inboundfeedhc': {u'checkpoint': None, u'is_source': False, u'state': 1}, u'mbus:slave:inboundaggregator': {u'checkpoint': None, u'is_source': False, u'state': 1}, u'mbus:slave:dshield_blocklist': {u'checkpoint': None, u'is_source': True, u'state': 1}, u'mbus:slave:inboundfeedlc': {u'checkpoint': None, u'is_source': False, u'state': 1}, u'mbus:slave:inboundfeedmc': {u'checkpoint': None, u'is_source': False, u'state': 1}, u'mbus:slave:spamhaus_EDROP': {u'checkpoint': None, u'is_source': True, u'state': 1}} 2019-07-06T20:22:30 (1191)mgmtbus.init_graph INFO: changes: [] 2019-07-06T20:22:30 (1191)startupplanner._plan_subgraph INFO: state_info: {u'spamhaus_EDROP': {u'checkpoint': None, u'is_source': True, u'state': 1}, u'dshield_blocklist': {u'checkpoint': None, u'is_source': True, u'state': 1}, u'inboundfeedlc': {u'checkpoint': None, u'is_source': False, u'state': 1}, u'inboundfeedhc': {u'checkpoint': None, u'is_source': False, u'state': 1}, u'spamhaus_DROP': {u'checkpoint': None, u'is_source': True, u'state': 1}, u'wlWhiteListIPv4': {u'checkpoint': None, u'is_source': True, u'state': 1}, u'inboundaggregator': {u'checkpoint': None, u'is_source': False, u'state': 1}, u'inboundfeedmc': {u'checkpoint': None, u'is_source': False, u'state': 1}} 2019-07-06T20:22:30 (1191)startupplanner._plan_subgraph INFO: planning for subgraph ['spamhaus_EDROP', 'dshield_blocklist', 'inboundfeedlc', 'inboundfeedhc', 'spamhaus_DROP', 'wlWhiteListIPv4', 'inboundaggregator', 'inboundfeedmc'] 2019-07-06T20:22:30 (1191)startupplanner._plan_subgraph INFO: No checkpoints, new graph: reset 2019-07-06T20:22:30 (1191)mgmtbus.init_graph INFO: spamhaus_EDROP <= reset 2019-07-06T20:22:30 (1203)base.state INFO: spamhaus_EDROP - transitioning to state 3 2019-07-06T20:22:31 (1203)base.state INFO: spamhaus_EDROP - transitioning to state 4 2019-07-06T20:22:31 (1191)mgmtbus.init_graph INFO: dshield_blocklist <= reset 2019-07-06T20:22:31 (1203)base.state INFO: dshield_blocklist - transitioning to state 3 2019-07-06T20:22:31 (1203)base.state INFO: dshield_blocklist - transitioning to state 4 2019-07-06T20:22:31 (1191)mgmtbus.init_graph INFO: inboundfeedlc <= reset 2019-07-06T20:22:31 (1203)base.state INFO: inboundfeedlc - transitioning to state 3 2019-07-06T20:22:31 (1203)base.state INFO: inboundfeedlc - transitioning to state 4 2019-07-06T20:22:31 (1191)mgmtbus.init_graph INFO: inboundfeedhc <= reset 2019-07-06T20:22:31 (1203)base.state INFO: inboundfeedhc - transitioning to state 3 2019-07-06T20:22:31 (1203)base.state INFO: inboundfeedhc - transitioning to state 4 2019-07-06T20:22:31 (1191)mgmtbus.init_graph INFO: spamhaus_DROP <= reset 2019-07-06T20:22:31 (1203)base.state INFO: spamhaus_DROP - transitioning to state 3 2019-07-06T20:22:31 (1203)base.state INFO: spamhaus_DROP - transitioning to state 4 2019-07-06T20:22:31 (1191)mgmtbus.init_graph INFO: wlWhiteListIPv4 <= reset 2019-07-06T20:22:31 (1203)base.state INFO: wlWhiteListIPv4 - transitioning to state 3 2019-07-06T20:22:31 (1203)base.state INFO: wlWhiteListIPv4 - transitioning to state 4 2019-07-06T20:22:31 (1191)mgmtbus.init_graph INFO: inboundaggregator <= reset 2019-07-06T20:22:31 (1203)base.state INFO: inboundaggregator - transitioning to state 3 2019-07-06T20:22:31 (1203)base.state INFO: inboundaggregator - transitioning to state 4 2019-07-06T20:22:31 (1191)mgmtbus.init_graph INFO: inboundfeedmc <= reset 2019-07-06T20:22:31 (1203)base.state INFO: inboundfeedmc - transitioning to state 3 2019-07-06T20:22:31 (1203)base.state INFO: inboundfeedmc - transitioning to state 4 2019-07-06T20:22:31 (1203)chassis.mgmtbus_start INFO: chassis - start received from mgmtbus 2019-07-06T20:22:31 (1203)chassis.start INFO: chassis start called 2019-07-06T20:22:31 (1203)base.state INFO: spamhaus_EDROP - transitioning to state 5 2019-07-06T20:22:31 (1203)base.state INFO: dshield_blocklist - transitioning to state 5 2019-07-06T20:22:31 (1203)base.state INFO: inboundaggregator - transitioning to state 5 2019-07-06T20:22:31 (1203)base.state INFO: inboundfeedhc - transitioning to state 5 2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_EDROP - command: 1562440951063 age_out 2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _age_out: 0 2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: dshield_blocklist - command: 1562440951064 age_out 2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _age_out: 0 2019-07-06T20:22:31 (1203)basepoller._huppable_wait INFO: hup is clear: False 2019-07-06T20:22:31 (1203)base.state INFO: spamhaus_DROP - transitioning to state 5 2019-07-06T20:22:31 (1203)base.state INFO: wlWhiteListIPv4 - transitioning to state 5 2019-07-06T20:22:31 (1203)base.state INFO: inboundfeedlc - transitioning to state 5 2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_EDROP - command: 1562440951065 poll 2019-07-06T20:22:31 (1203)basepoller._polling_loop INFO: Polling spamhaus_EDROP 2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_DROP - command: 1562440951077 age_out 2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _age_out: 0 2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: wlWhiteListIPv4 - command: 1562440951077 age_out 2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _age_out: 0 2019-07-06T20:22:31 (1203)basepoller._huppable_wait INFO: hup is clear: False 2019-07-06T20:22:31 (1203)base.state INFO: inboundfeedmc - transitioning to state 5 2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_DROP - command: 1562440951078 poll 2019-07-06T20:22:31 (1203)basepoller._polling_loop INFO: Polling spamhaus_DROP 2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_EDROP - command: 1562440951065 sudden_death 2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _last_run: 0 2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_EDROP - command: 1562440951065 age_out 2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _age_out: 0 2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_EDROP - command: 1562440951065 gc 2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _withdrawn: 0 2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_DROP - command: 1562440951078 sudden_death 2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _last_run: 0 2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_DROP - command: 1562440951078 age_out 2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _age_out: 0 2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_DROP - command: 1562440951078 gc 2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _withdrawn: 0 2019-07-06T20:22:32 (1203)launcher._run_chassis INFO: Nodes initialized 2019-07-06T20:22:33 (1203)basepoller._huppable_wait INFO: hup is clear: False 2019-07-06T20:22:33 (1203)basepoller._huppable_wait INFO: hup is clear: False 2019-07-06T20:22:33 (1203)basepoller._actor_loop INFO: dshield_blocklist - command: 1562440953183 poll 2019-07-06T20:22:33 (1203)basepoller._polling_loop INFO: Polling dshield_blocklist 2019-07-06T20:22:33 (1203)basepoller._actor_loop INFO: wlWhiteListIPv4 - command: 1562440953183 poll 2019-07-06T20:22:33 (1203)basepoller._polling_loop INFO: Polling wlWhiteListIPv4 2019-07-06T20:22:33 (1203)basepoller._actor_loop INFO: wlWhiteListIPv4 - command: 1562440953183 sudden_death 2019-07-06T20:22:33 (1203)basepoller._actor_loop INFO: wlWhiteListIPv4 - command: 1562440953183 age_out 2019-07-06T20:22:33 (1203)table._query_by_index INFO: Deleted in scan of _age_out: 0 2019-07-06T20:22:33 (1203)basepoller._actor_loop INFO: wlWhiteListIPv4 - command: 1562440953183 gc 2019-07-06T20:22:33 (1203)table._query_by_index INFO: Deleted in scan of _withdrawn: 0 2019-07-06T20:22:34 (1203)basepoller._actor_loop INFO: dshield_blocklist - command: 1562440953183 sudden_death 2019-07-06T20:22:34 (1203)table._query_by_index INFO: Deleted in scan of _last_run: 0 2019-07-06T20:22:34 (1203)basepoller._actor_loop INFO: dshield_blocklist - command: 1562440953183 age_out 2019-07-06T20:22:34 (1203)table._query_by_index INFO: Deleted in scan of _age_out: 0 2019-07-06T20:22:34 (1203)basepoller._actor_loop INFO: dshield_blocklist - command: 1562440953183 gc 2019-07-06T20:22:34 (1203)table._query_by_index INFO: Deleted in scan of _withdrawn: 0
@LarsAtConsigas please open a discussion with your logs, and please check that redis is running.
Luigi
I am trying to install Minemeld on Ubuntu 16.04. I ran through all the above steps, restarted, and ran the below command to check if it is running:
sudo -u minemeld /opt/minemeld/engine/current/bin/supervisorctl -c /opt/minemeld/local/supervisor/config/supervisord.conf status
I get the error:
unix:///var/run/minemeld/minemeld.sock no such file
When I try to start the service with:
service minemeld start
I get the error:
Failed to start minemeld.service: Unit redis.service not found.
I have installed redis and checked with:
service redis-server status
I get:
● redis-server.service - Advanced key-value store Loaded: loaded (/lib/systemd/system/redis-server.service; disabled; vendor preset: disabled) Active: active (running) since Sun 2019-07-07 17:30:19 PDT; 20h ago Docs: http://redis.io/documentation, man:redis-server(1) Main PID: 1278 (redis-server) Tasks: 3 Memory: 1.8M CPU: 1min 14.907s CGroup: /system.slice/redis-server.service └─1278 /usr/bin/redis-server 127.0.0.1:6379
I have also ran the command:
sudo /usr/sbin/minemeld-auto-update
And get:
2019-07-08 14:01:34,630 INFO:0.9.12 Current status: 2019-07-08 14:01:34,631 INFO:0.9.12 minemeld-engine: current: 0.9.62 latest: 0.9.62 2019-07-08 14:01:34,631 INFO:0.9.12 minemeld-webui: current: 0.9.62 latest: 0.9.62 2019-07-08 14:01:34,631 INFO:0.9.12 minemeld-prototypes: current: 0.9.62 latest: 0.9.62 2019-07-08 14:01:34,678 DEBUG:0.9.12 curl output: 2019-07-08 14:01:34,727 DEBUG:0.9.12 curl output: 2019-07-08 14:01:34,727 DEBUG:0.9.12 gpgv: /usr/bin/gpgv --ignore-time-conflict --keyring /etc/apt/trusted.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-jessie-stable.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-jessie-security-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-jessie-automatic.gpg --keyring /etc/apt/trusted.gpg.d/minemeld.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-stretch-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-buster-stable.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-buster-security-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-buster-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-stretch-stable.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-stretch-security-automatic.gpg /tmp/mmaupackagesgpgnI2ehE /tmp/mmaupackagesahpjtz 2019-07-08 14:01:34,731 INFO:0.9.12 gpgv output: gpgv: Signature made Thu 27 Jun 2019 04:54:07 AM PDT using RSA key ID 7B630999 gpgv: Good signature from "Palo Alto Networks, MineMeld Team <minemeld@paloaltonetworks.com>" gpgv: aka "[invalid image]" 2019-07-08 14:01:34,734 INFO:0.9.12 No package to deploy, exit
Please advise.
Thanks.
Hello all,
I tried to install MineMeld, but at Checking if MineMeld is running I got this error:
sudo -u minemeld /opt/minemeld/engine/current/bin/supervisorctl -c /opt/minemeld/local/supervisor/config/supervisord.conf status
sudo: /opt/minemeld/engine/current/bin/supervisorctl: command not found
How can I solve this ?
Regards
Currently, Ubuntu 19.x is the server version while LTS is at 18.x. Are there instructions for installing MineMeld on current Ubuntu releases?
Thank you.
Reference: https://ubuntu.com/download/server
FWIW, I followed the instructions provided by Lmori on Ubuntu 18.04.3 LTS server, patched with the latest updates as of the time of this post. After the reboot, MineMeld was not listed as a service. I reran the following at which point MineMeld was installed as a service.
sudo apt install -o Dpkg::Options::="--force-overwrite" -y minemeld
I browsed to the server on port 443 and was presented with a login to MineMeld. I login as admin/minemeld but received "Bad Gateway". The log files show the following, which are the same as logs others have posted:
[2019-09-04 01:18:17 +0000] [3499] [ERROR] Exception in worker process Traceback (most recent call last): File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 517, in spawn_worker worker.init_process() File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 190, in init_process super(GeventWorker, self).init_process() File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 122, in init_process self.load_wsgi() File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 132, in load_wsgi self.wsgi = self.app.wsgi() File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi self.callable = self.load() File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load return self.load_wsgiapp() File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp return util.import_app(self.app_uri) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/gunicorn/util.py", line 357, in import_app __import__(module) File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/flask/main.py", line 3, in <module> app = create_app() File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/flask/__init__.py", line 67, in create_app from . import metricsapi # noqa File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/flask/metricsapi.py", line 19, in <module> import rrdtool ImportError: librrd.so.4: cannot open shared object file: No such file or directory [2019-09-04 01:18:17 +0000] [3499] [INFO] Worker exiting (pid: 3499) [2019-09-04 01:18:17 +0000] [3496] [INFO] Shutting down: Master [2019-09-04 01:18:17 +0000] [3496] [INFO] Reason: Worker failed to boot.
I have found many posts about this issue, yet no solutions. rrdtool itself does exist in the minemeld folder.
Any advice?
Thank you.
Is this the preferred method to install Minemeld or should I use the anisible method?
Apart from the linux hardedning and the repository, will the end results be the same?
Hi @Potato-soup,
this mechanism also supports auto updates. I am working on an article on how to use Docker to install MineMeld, that is the favorite mechanism.
@Cthroop, are you using a 64bit Ubuntu instance? Could you open a discussion to check your issues?