This website uses cookies essential to its operation, for analytics, and for personalized content. By continuing to browse this site, you acknowledge the use of cookies.
For details on cookie usage on our site, read our Privacy Policy
For details on cookie usage on our site, read our Privacy Policy
About LarsAtConsigas
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
- LIVEcommunity
- About LarsAtConsigas
08-25-2020
55Posts
13Likes
0Solutions
Aug 25, 2020Last Visited
User
Activity
User
Profile
Latest posts by LarsAtConsigas
| Subject | Views | Posted |
|---|---|---|
| 4429 | 06-30-2020 10:34 PM | |
| 1828 | 06-30-2020 01:40 AM | |
| 2239 | 06-30-2020 01:40 AM | |
| 6903 | 06-30-2020 12:22 AM | |
| 10227 | 06-30-2020 12:19 AM | |
| 6172 | 07-10-2019 12:07 AM | |
| 6201 | 07-08-2019 11:34 PM | |
| 6380 | 07-08-2019 09:22 AM | |
| 6389 | 07-08-2019 08:53 AM | |
| 6486 | 07-07-2019 02:56 PM |
My Liked Posts
| Subject | Likes | Posted |
|---|---|---|
| 1 Like | 06-30-2020 12:19 AM | |
| 3 Likes | 06-30-2020 01:40 AM | |
| 1 Like | 03-26-2014 12:04 AM |
User Badges
Community Statistics
| Member Since | 07-05-2012 11:00 AM |
| Date Last Visited | 08-25-2020 05:17 AM |
| Posts | 55 |
| Total Likes Received | 5 |
06-30-2020
10:34 PM
Hi @JohnQuile , I haven't worked with Okta but I imagine that the steps are very similar.
... View more
06-30-2020
01:40 AM
3 Kudos
We posted a training video explaining how to securely set up SAML authentication with GlobalProtect end-to-end against Office 365 Azure AD. At 29:35 it shows how to enable "Validate Identity Provider Certificate" and fix the commit error "Validate Identity Provider Certificate is checked but no Certificate Profile is provided authentication-profile"
... View more
06-30-2020
01:40 AM
We posted a training video explaining how to securely set up SAML authentication on Prisma Access end-to-end against Office 365 Azure AD. At 29:35 it shows how to enable "Validate Identity Provider Certificate" and fix the commit error "Validate Identity Provider Certificate is checked but no Certificate Profile is provided authentication-profile"
... View more
06-30-2020
12:22 AM
We posted a training video explaining how to securely set up SAML authentication end-to-end against Office 365 Azure AD. The critical element which explains how to set up certificate validation of the SAML Identity Provider to address the SAML Bypass Vulnerability (CVE-2020-2021) starts at 29:35. It shows how to enable "Validate Identity Provider Certificate" and fix the commit error "Validate Identity Provider Certificate is checked but no Certificate Profile is provided authentication-profile"
... View more
06-30-2020
12:19 AM
1 Kudo
We posted a training video explaining how to securely set up SAML authentication end-to-end against Office 365 Azure AD. The critical element which explains how to set up certificate validation of the SAML Identity Provider to address the SAML Bypass Vulnerability (CVE-2020-2021) starts at 29:35. It shows how to enable "Validate Identity Provider Certificate" and fix the commit error "Validate Identity Provider Certificate is checked but no Certificate Profile is provided authentication-profile"
... View more
07-10-2019
12:07 AM
Thanks Luigi, turns out that the standard Ubuntu install didn't link the hardware clock with the system clock and the hardware clock was 2h in the future. The minemeld engine now starts imidiately after fixing this.
... View more
07-08-2019
11:34 PM
After a couple of hours the minemeld-engine process shows as RUNNING. Any idea what is causing this long delay?
... View more
07-08-2019
09:22 AM
I checked the status after the server has been up all day and to my surprise it showed as RUNNING. So I restarted the server again and now after over 20min the status still shows as STARTING
I cannot find any obvious errors in minemeld-engine.log but there is a strange timeshift
2019-07-08T19:02:11 (1210)basepoller._actor_loop INFO: dshield_blocklist - command: 1562608931147 age_out
2019-07-08T19:02:11 (1210)table._query_by_index INFO: Deleted in scan of _age_out: 0
2019-07-08T19:02:11 (1210)basepoller._actor_loop INFO: dshield_blocklist - command: 1562608931147 gc
2019-07-08T19:02:11 (1210)table._query_by_index INFO: Deleted in scan of _withdrawn: 0
2019-07-08T17:03:16 (1210)basepoller._actor_loop INFO: wlWhiteListIPv4 - command: 1562601796946 age_out
2019-07-08T17:03:16 (1210)basepoller._actor_loop INFO: wlWhiteListIPv4 - command age_out, old timestamp - ignored
2019-07-08T17:04:23 (1210)basepoller._actor_loop INFO: wlWhiteListIPv4 - command: 1562601863949 age_out
2019-07-08T17:04:23 (1210)basepoller._actor_loop INFO: wlWhiteListIPv4 - command age_out, old timestamp - ignored
2019-07-08T17:05:30 (1210)basepoller._actor_loop INFO: wlWhiteListIPv4 - command: 1562601930950 age_out
and the other processes show a negative uptime
lab-user@minemeld:~$ sudo -u minemeld /opt/minemeld/engine/current/bin/supervisorctl -c /opt/minemeld/local/supervisor/config/supervisord.conf status
minemeld-engine STARTING
minemeld-supervisord-listener RUNNING pid 1195, uptime -1 day, 22:15:41
minemeld-traced RUNNING pid 1197, uptime -1 day, 22:15:41
minemeld-web RUNNING pid 1198, uptime -1 day, 22:15:41
... View more
07-08-2019
08:53 AM
Hi Luigi,
yes these two entires are in redis.conf
Cheers
Lars
... View more
07-07-2019
02:56 PM
I installed Minemeld on a fresh Ubuntu 16.04.6 LTS with the instructions "Manually Install MineMeld On Ubuntu Server 16.04" but the minemeld-engine is hanging in STARTING state. minemeld-engine.log shows a couple of directory not found messages for CheckPoints. Any idea how to fix this?
lab-user@minemeld:~$ sudo -u minemeld /opt/minemeld/engine/current/bin/supervisorctl -c /opt/minemeld/local/supervisor/config/supervisord.conf status
[sudo] password for lab-user:
minemeld-engine STARTING
minemeld-supervisord-listener RUNNING pid 1190, uptime 0:00:19
minemeld-traced RUNNING pid 1192, uptime 0:00:19
minemeld-web RUNNING pid 1193, uptime 0:00:19
redis is running as well
lab-user@minemeld:~$ sudo systemctl status redis
● redis-server.service - Advanced key-value store
Loaded: loaded (/lib/systemd/system/redis-server.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2019-07-08 00:44:38 IST; 1h 53min left
Docs: http://redis.io/documentation,
man:redis-server(1)
Process: 1069 ExecStartPost=/bin/run-parts --verbose /etc/redis/redis-server.post-up.d (code=exited, status=0/SUCCESS)
Process: 1032 ExecStart=/usr/bin/redis-server /etc/redis/redis.conf (code=exited, status=0/SUCCESS)
Process: 1006 ExecStartPre=/bin/run-parts --verbose /etc/redis/redis-server.pre-up.d (code=exited, status=0/SUCCESS)
Main PID: 1068 (redis-server)
Tasks: 3
Memory: 2.8M
CPU: 589ms
CGroup: /system.slice/redis-server.service
└─1068 /usr/bin/redis-server 127.0.0.1:6379
Jul 08 00:44:37 minemeld systemd[1]: Starting Advanced key-value store...
Jul 08 00:44:37 minemeld run-parts[1006]: run-parts: executing /etc/redis/redis-server.pre-up.d/00_example
Jul 08 00:44:38 minemeld systemd[1]: Started Advanced key-value store.
... View more
07-07-2019
02:36 PM
@lmori yes I did, but unfortunately it didn't help.
... View more
07-06-2019
10:31 AM
I installed Minemeld on a fresh Ubuntu 16.04.6 LTS with the instructions above but the minemeld-engine is hanging in STARTING state. minemeld-engine.log shows a couple of directory not found messages for CheckPoints. Any idea how to fix this?
lab-user@minemeld:~$ sudo -u minemeld /opt/minemeld/engine/current/bin/supervisorctl -c /opt/minemeld/local/supervisor/config/supervisord.conf status
[sudo] password for lab-user:
minemeld-engine STARTING
minemeld-supervisord-listener RUNNING pid 1190, uptime 0:00:19
minemeld-traced RUNNING pid 1192, uptime 0:00:19
minemeld-web RUNNING pid 1193, uptime 0:00:19
2019-07-06T18:21:20 (4767)launcher.main INFO: Starting mm-run.py version 0.9.62
2019-07-06T18:21:20 (4767)launcher.main INFO: mm-run.py arguments: Namespace(config='/opt/minemeld/local/config', multiprocessing=0, nodes_per_chassis=15.0, verbose=False)
2019-07-06T18:21:21 (4767)config._load_config_from_dir INFO: Switching to candidate config
2019-07-06T18:21:21 (4767)config._load_config_from_dir INFO: Changes in config: [_ConfigChange(nodename='spamhaus_EDROP', nodeclass='minemeld.ft.http.HttpFT', change=0, detail=None), _ConfigChange(nodename='dshield_blocklist', nodeclass='minemeld.ft.http.HttpFT', change=0, detail=None), _ConfigChange(nodename='inboundaggregator', nodeclass='minemeld.ft.ipop.AggregateIPv4FT', change=0, detail=None), _ConfigChange(nodename='inboundfeedhc', nodeclass='minemeld.ft.redis.RedisSet', change=0, detail=None), _ConfigChange(nodename='spamhaus_DROP', nodeclass='minemeld.ft.http.HttpFT', change=0, detail=None), _ConfigChange(nodename='wlWhiteListIPv4', nodeclass='minemeld.ft.local.YamlIPv4FT', change=0, detail=None), _ConfigChange(nodename='inboundfeedlc', nodeclass='minemeld.ft.redis.RedisSet', change=0, detail=None), _ConfigChange(nodename='inboundfeedmc', nodeclass='minemeld.ft.redis.RedisSet', change=0, detail=None)]
2019-07-06T18:21:21 (4767)config._destroy_old_nodes INFO: Destroyed nodes: []
2019-07-06T18:21:21 (4767)launcher.main INFO: mm-run.py config: _Config(nodes={'spamhaus_EDROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.EDROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/edrop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'dshield_blocklist': {'output': True, 'config': {'indicator': {'regex': '^([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})\\t([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})', 'transform': '\\1-\\2'}, 'source_name': 'dshield.block', 'age_out': {'default': None, 'sudden_death': True, 'interval': 257}, 'url': 'https://www.dshield.org/block.txt', 'fields': {'dshield_name': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t([^\\t]+)', 'transform': '\\1'}, 'dshield_country': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t([A-Z]+)', 'transform': '\\1'}, 'dshield_nattacks': {'regex': '^.*\\t.*\\t[0-9]+\\t([0-9]+)', 'transform': '\\1'}, 'dshield_email': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t[A-Z]+\\t(\\S+)', 'transform': '\\1'}}, 'interval': 619, 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '[#S].*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'inboundaggregator': {'inputs': ['spamhaus_DROP', 'spamhaus_EDROP', 'dshield_blocklist', 'wlWhiteListIPv4'], 'config': {'whitelist_prefixes': ['wl'], 'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", "direction == 'inbound'"], 'name': 'accept inbound IPv4', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", 'direction == null'], 'name': 'accept generic IPv4', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.ipop.AggregateIPv4FT', 'output': True}, 'inboundfeedhc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence > 75', "share_level == 'green'"], 'name': 'accept confidence > 75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'spamhaus_DROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.DROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/drop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'wlWhiteListIPv4': {'inputs': [], 'config': {'attributes': {'confidence': 100, 'share_level': 'red'}, 'interval': 3600, 'age_out': {'default': None, 'sudden_death': True, 'interval': 67}}, 'class': 'minemeld.ft.local.YamlIPv4FT', 'output': True}, 'inboundfeedlc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence < 50', "share_level == 'green'"], 'name': 'accept confidence < 50 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'inboundfeedmc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence >= 50', 'confidence < 75', "share_level == 'green'"], 'name': 'accept confidence 50-75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}}, fabric={'config': {'priority': -2, 'num_connections': 50}, 'class': 'ZMQRedis'}, mgmtbus={'slave': {}, 'master': {}, 'transport': {'config': {'priority': 2, 'num_connections': 10}, 'class': 'ZMQRedis'}}, changes=[_ConfigChange(nodename='spamhaus_EDROP', nodeclass='minemeld.ft.http.HttpFT', change=0, detail=None), _ConfigChange(nodename='dshield_blocklist', nodeclass='minemeld.ft.http.HttpFT', change=0, detail=None), _ConfigChange(nodename='inboundaggregator', nodeclass='minemeld.ft.ipop.AggregateIPv4FT', change=0, detail=None), _ConfigChange(nodename='inboundfeedhc', nodeclass='minemeld.ft.redis.RedisSet', change=0, detail=None), _ConfigChange(nodename='spamhaus_DROP', nodeclass='minemeld.ft.http.HttpFT', change=0, detail=None), _ConfigChange(nodename='wlWhiteListIPv4', nodeclass='minemeld.ft.local.YamlIPv4FT', change=0, detail=None), _ConfigChange(nodename='inboundfeedlc', nodeclass='minemeld.ft.redis.RedisSet', change=0, detail=None), _ConfigChange(nodename='inboundfeedmc', nodeclass='minemeld.ft.redis.RedisSet', change=0, detail=None)])
2019-07-06T18:21:21 (4767)launcher.main INFO: multiprocessing: #cores: 2
2019-07-06T18:21:21 (4767)launcher.main INFO: multiprocessing: max #chassis: 2
2019-07-06T18:21:21 (4767)launcher.main INFO: Number of chassis: 1
Traceback (most recent call last):
File "/opt/minemeld/engine/current/bin/mm-run", line 10, in <module>
sys.exit(main())
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 252, in main
minemeld.comm.cleanup(config.fabric['class'], config.fabric['config'])
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/__init__.py", line 14, in cleanup
return ZMQRedis.cleanup(config)
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/zmqredis.py", line 855, in cleanup
tkeys = SR.keys(pattern='mm:topic:*')
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 936, in keys
return self.execute_command('KEYS', pattern)
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 578, in execute_command
connection.send_command(*args)
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 563, in send_command
self.send_packed_command(self.pack_command(*args))
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 538, in send_packed_command
self.connect()
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 442, in connect
raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 2 connecting to unix socket: /var/run/redis/redis.sock. No such file or directory.
2019-07-06T18:21:33 (4953)launcher.main INFO: Starting mm-run.py version 0.9.62
2019-07-06T18:21:33 (4953)launcher.main INFO: mm-run.py arguments: Namespace(config='/opt/minemeld/local/config', multiprocessing=0, nodes_per_chassis=15.0, verbose=False)
2019-07-06T18:21:35 (4953)config._load_config_from_dir INFO: Switching to candidate config
2019-07-06T18:21:35 (4953)config._load_config_from_dir INFO: Changes in config: []
2019-07-06T18:21:35 (4953)config._destroy_old_nodes INFO: Destroyed nodes: []
2019-07-06T18:21:35 (4953)launcher.main INFO: mm-run.py config: _Config(nodes={'spamhaus_EDROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.EDROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/edrop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'dshield_blocklist': {'output': True, 'config': {'indicator': {'regex': '^([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})\\t([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})', 'transform': '\\1-\\2'}, 'source_name': 'dshield.block', 'age_out': {'default': None, 'sudden_death': True, 'interval': 257}, 'url': 'https://www.dshield.org/block.txt', 'fields': {'dshield_name': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t([^\\t]+)', 'transform': '\\1'}, 'dshield_country': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t([A-Z]+)', 'transform': '\\1'}, 'dshield_nattacks': {'regex': '^.*\\t.*\\t[0-9]+\\t([0-9]+)', 'transform': '\\1'}, 'dshield_email': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t[A-Z]+\\t(\\S+)', 'transform': '\\1'}}, 'interval': 619, 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '[#S].*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'inboundaggregator': {'inputs': ['spamhaus_DROP', 'spamhaus_EDROP', 'dshield_blocklist', 'wlWhiteListIPv4'], 'config': {'whitelist_prefixes': ['wl'], 'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", "direction == 'inbound'"], 'name': 'accept inbound IPv4', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", 'direction == null'], 'name': 'accept generic IPv4', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.ipop.AggregateIPv4FT', 'output': True}, 'inboundfeedhc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence > 75', "share_level == 'green'"], 'name': 'accept confidence > 75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'spamhaus_DROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.DROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/drop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'wlWhiteListIPv4': {'inputs': [], 'config': {'attributes': {'confidence': 100, 'share_level': 'red'}, 'interval': 3600, 'age_out': {'default': None, 'sudden_death': True, 'interval': 67}}, 'class': 'minemeld.ft.local.YamlIPv4FT', 'output': True}, 'inboundfeedlc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence < 50', "share_level == 'green'"], 'name': 'accept confidence < 50 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'inboundfeedmc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence >= 50', 'confidence < 75', "share_level == 'green'"], 'name': 'accept confidence 50-75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}}, fabric={'config': {'priority': -2, 'num_connections': 50}, 'class': 'ZMQRedis'}, mgmtbus={'slave': {}, 'master': {}, 'transport': {'config': {'priority': 2, 'num_connections': 10}, 'class': 'ZMQRedis'}}, changes=[])
2019-07-06T18:21:35 (4953)launcher.main INFO: multiprocessing: #cores: 2
2019-07-06T18:21:35 (4953)launcher.main INFO: multiprocessing: max #chassis: 2
2019-07-06T18:21:35 (4953)launcher.main INFO: Number of chassis: 1
Traceback (most recent call last):
File "/opt/minemeld/engine/current/bin/mm-run", line 10, in <module>
sys.exit(main())
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 252, in main
minemeld.comm.cleanup(config.fabric['class'], config.fabric['config'])
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/__init__.py", line 14, in cleanup
return ZMQRedis.cleanup(config)
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/zmqredis.py", line 855, in cleanup
tkeys = SR.keys(pattern='mm:topic:*')
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 936, in keys
return self.execute_command('KEYS', pattern)
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 578, in execute_command
connection.send_command(*args)
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 563, in send_command
self.send_packed_command(self.pack_command(*args))
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 538, in send_packed_command
self.connect()
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 442, in connect
raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 2 connecting to unix socket: /var/run/redis/redis.sock. No such file or directory.
2019-07-06T18:21:36 (4969)launcher.main INFO: Starting mm-run.py version 0.9.62
2019-07-06T18:21:36 (4969)launcher.main INFO: mm-run.py arguments: Namespace(config='/opt/minemeld/local/config', multiprocessing=0, nodes_per_chassis=15.0, verbose=False)
2019-07-06T18:21:38 (4969)config._load_config_from_dir INFO: Switching to candidate config
2019-07-06T18:21:38 (4969)config._load_config_from_dir INFO: Changes in config: []
2019-07-06T18:21:38 (4969)config._destroy_old_nodes INFO: Destroyed nodes: []
2019-07-06T18:21:38 (4969)launcher.main INFO: mm-run.py config: _Config(nodes={'spamhaus_EDROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.EDROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/edrop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'dshield_blocklist': {'output': True, 'config': {'indicator': {'regex': '^([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})\\t([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})', 'transform': '\\1-\\2'}, 'source_name': 'dshield.block', 'age_out': {'default': None, 'sudden_death': True, 'interval': 257}, 'url': 'https://www.dshield.org/block.txt', 'fields': {'dshield_name': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t([^\\t]+)', 'transform': '\\1'}, 'dshield_country': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t([A-Z]+)', 'transform': '\\1'}, 'dshield_nattacks': {'regex': '^.*\\t.*\\t[0-9]+\\t([0-9]+)', 'transform': '\\1'}, 'dshield_email': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t[A-Z]+\\t(\\S+)', 'transform': '\\1'}}, 'interval': 619, 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '[#S].*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'inboundaggregator': {'inputs': ['spamhaus_DROP', 'spamhaus_EDROP', 'dshield_blocklist', 'wlWhiteListIPv4'], 'config': {'whitelist_prefixes': ['wl'], 'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", "direction == 'inbound'"], 'name': 'accept inbound IPv4', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", 'direction == null'], 'name': 'accept generic IPv4', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.ipop.AggregateIPv4FT', 'output': True}, 'inboundfeedhc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence > 75', "share_level == 'green'"], 'name': 'accept confidence > 75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'spamhaus_DROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.DROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/drop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'wlWhiteListIPv4': {'inputs': [], 'config': {'attributes': {'confidence': 100, 'share_level': 'red'}, 'interval': 3600, 'age_out': {'default': None, 'sudden_death': True, 'interval': 67}}, 'class': 'minemeld.ft.local.YamlIPv4FT', 'output': True}, 'inboundfeedlc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence < 50', "share_level == 'green'"], 'name': 'accept confidence < 50 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'inboundfeedmc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence >= 50', 'confidence < 75', "share_level == 'green'"], 'name': 'accept confidence 50-75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}}, fabric={'config': {'priority': -2, 'num_connections': 50}, 'class': 'ZMQRedis'}, mgmtbus={'slave': {}, 'master': {}, 'transport': {'config': {'priority': 2, 'num_connections': 10}, 'class': 'ZMQRedis'}}, changes=[])
2019-07-06T18:21:38 (4969)launcher.main INFO: multiprocessing: #cores: 2
2019-07-06T18:21:38 (4969)launcher.main INFO: multiprocessing: max #chassis: 2
2019-07-06T18:21:38 (4969)launcher.main INFO: Number of chassis: 1
Traceback (most recent call last):
File "/opt/minemeld/engine/current/bin/mm-run", line 10, in <module>
sys.exit(main())
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 252, in main
minemeld.comm.cleanup(config.fabric['class'], config.fabric['config'])
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/__init__.py", line 14, in cleanup
return ZMQRedis.cleanup(config)
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/zmqredis.py", line 855, in cleanup
tkeys = SR.keys(pattern='mm:topic:*')
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 936, in keys
return self.execute_command('KEYS', pattern)
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 578, in execute_command
connection.send_command(*args)
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 563, in send_command
self.send_packed_command(self.pack_command(*args))
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 538, in send_packed_command
self.connect()
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 442, in connect
raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 2 connecting to unix socket: /var/run/redis/redis.sock. No such file or directory.
2019-07-06T18:21:40 (4974)launcher.main INFO: Starting mm-run.py version 0.9.62
2019-07-06T18:21:40 (4974)launcher.main INFO: mm-run.py arguments: Namespace(config='/opt/minemeld/local/config', multiprocessing=0, nodes_per_chassis=15.0, verbose=False)
2019-07-06T18:21:42 (4974)config._load_config_from_dir INFO: Switching to candidate config
2019-07-06T18:21:42 (4974)config._load_config_from_dir INFO: Changes in config: []
2019-07-06T18:21:42 (4974)config._destroy_old_nodes INFO: Destroyed nodes: []
2019-07-06T18:21:42 (4974)launcher.main INFO: mm-run.py config: _Config(nodes={'spamhaus_EDROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.EDROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/edrop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'dshield_blocklist': {'output': True, 'config': {'indicator': {'regex': '^([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})\\t([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})', 'transform': '\\1-\\2'}, 'source_name': 'dshield.block', 'age_out': {'default': None, 'sudden_death': True, 'interval': 257}, 'url': 'https://www.dshield.org/block.txt', 'fields': {'dshield_name': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t([^\\t]+)', 'transform': '\\1'}, 'dshield_country': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t([A-Z]+)', 'transform': '\\1'}, 'dshield_nattacks': {'regex': '^.*\\t.*\\t[0-9]+\\t([0-9]+)', 'transform': '\\1'}, 'dshield_email': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t[A-Z]+\\t(\\S+)', 'transform': '\\1'}}, 'interval': 619, 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '[#S].*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'inboundaggregator': {'inputs': ['spamhaus_DROP', 'spamhaus_EDROP', 'dshield_blocklist', 'wlWhiteListIPv4'], 'config': {'whitelist_prefixes': ['wl'], 'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", "direction == 'inbound'"], 'name': 'accept inbound IPv4', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", 'direction == null'], 'name': 'accept generic IPv4', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.ipop.AggregateIPv4FT', 'output': True}, 'inboundfeedhc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence > 75', "share_level == 'green'"], 'name': 'accept confidence > 75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'spamhaus_DROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.DROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/drop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'wlWhiteListIPv4': {'inputs': [], 'config': {'attributes': {'confidence': 100, 'share_level': 'red'}, 'interval': 3600, 'age_out': {'default': None, 'sudden_death': True, 'interval': 67}}, 'class': 'minemeld.ft.local.YamlIPv4FT', 'output': True}, 'inboundfeedlc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence < 50', "share_level == 'green'"], 'name': 'accept confidence < 50 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'inboundfeedmc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence >= 50', 'confidence < 75', "share_level == 'green'"], 'name': 'accept confidence 50-75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}}, fabric={'config': {'priority': -2, 'num_connections': 50}, 'class': 'ZMQRedis'}, mgmtbus={'slave': {}, 'master': {}, 'transport': {'config': {'priority': 2, 'num_connections': 10}, 'class': 'ZMQRedis'}}, changes=[])
2019-07-06T18:21:42 (4974)launcher.main INFO: multiprocessing: #cores: 2
2019-07-06T18:21:42 (4974)launcher.main INFO: multiprocessing: max #chassis: 2
2019-07-06T18:21:42 (4974)launcher.main INFO: Number of chassis: 1
Traceback (most recent call last):
File "/opt/minemeld/engine/current/bin/mm-run", line 10, in <module>
sys.exit(main())
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 252, in main
minemeld.comm.cleanup(config.fabric['class'], config.fabric['config'])
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/__init__.py", line 14, in cleanup
return ZMQRedis.cleanup(config)
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/zmqredis.py", line 855, in cleanup
tkeys = SR.keys(pattern='mm:topic:*')
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 936, in keys
return self.execute_command('KEYS', pattern)
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 578, in execute_command
connection.send_command(*args)
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 563, in send_command
self.send_packed_command(self.pack_command(*args))
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 538, in send_packed_command
self.connect()
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 442, in connect
raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 2 connecting to unix socket: /var/run/redis/redis.sock. No such file or directory.
2019-07-06T18:21:46 (4979)launcher.main INFO: Starting mm-run.py version 0.9.62
2019-07-06T18:21:46 (4979)launcher.main INFO: mm-run.py arguments: Namespace(config='/opt/minemeld/local/config', multiprocessing=0, nodes_per_chassis=15.0, verbose=False)
2019-07-06T18:21:47 (4979)config._load_config_from_dir INFO: Switching to candidate config
2019-07-06T18:21:47 (4979)config._load_config_from_dir INFO: Changes in config: []
2019-07-06T18:21:47 (4979)config._destroy_old_nodes INFO: Destroyed nodes: []
2019-07-06T18:21:47 (4979)launcher.main INFO: mm-run.py config: _Config(nodes={'spamhaus_EDROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.EDROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/edrop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'dshield_blocklist': {'output': True, 'config': {'indicator': {'regex': '^([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})\\t([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})', 'transform': '\\1-\\2'}, 'source_name': 'dshield.block', 'age_out': {'default': None, 'sudden_death': True, 'interval': 257}, 'url': 'https://www.dshield.org/block.txt', 'fields': {'dshield_name': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t([^\\t]+)', 'transform': '\\1'}, 'dshield_country': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t([A-Z]+)', 'transform': '\\1'}, 'dshield_nattacks': {'regex': '^.*\\t.*\\t[0-9]+\\t([0-9]+)', 'transform': '\\1'}, 'dshield_email': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t[A-Z]+\\t(\\S+)', 'transform': '\\1'}}, 'interval': 619, 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '[#S].*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'inboundaggregator': {'inputs': ['spamhaus_DROP', 'spamhaus_EDROP', 'dshield_blocklist', 'wlWhiteListIPv4'], 'config': {'whitelist_prefixes': ['wl'], 'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", "direction == 'inbound'"], 'name': 'accept inbound IPv4', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", 'direction == null'], 'name': 'accept generic IPv4', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.ipop.AggregateIPv4FT', 'output': True}, 'inboundfeedhc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence > 75', "share_level == 'green'"], 'name': 'accept confidence > 75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'spamhaus_DROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.DROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/drop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'wlWhiteListIPv4': {'inputs': [], 'config': {'attributes': {'confidence': 100, 'share_level': 'red'}, 'interval': 3600, 'age_out': {'default': None, 'sudden_death': True, 'interval': 67}}, 'class': 'minemeld.ft.local.YamlIPv4FT', 'output': True}, 'inboundfeedlc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence < 50', "share_level == 'green'"], 'name': 'accept confidence < 50 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'inboundfeedmc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence >= 50', 'confidence < 75', "share_level == 'green'"], 'name': 'accept confidence 50-75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}}, fabric={'config': {'priority': -2, 'num_connections': 50}, 'class': 'ZMQRedis'}, mgmtbus={'slave': {}, 'master': {}, 'transport': {'config': {'priority': 2, 'num_connections': 10}, 'class': 'ZMQRedis'}}, changes=[])
2019-07-06T18:21:47 (4979)launcher.main INFO: multiprocessing: #cores: 2
2019-07-06T18:21:47 (4979)launcher.main INFO: multiprocessing: max #chassis: 2
2019-07-06T18:21:47 (4979)launcher.main INFO: Number of chassis: 1
Traceback (most recent call last):
File "/opt/minemeld/engine/current/bin/mm-run", line 10, in <module>
sys.exit(main())
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 252, in main
minemeld.comm.cleanup(config.fabric['class'], config.fabric['config'])
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/__init__.py", line 14, in cleanup
return ZMQRedis.cleanup(config)
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/comm/zmqredis.py", line 855, in cleanup
tkeys = SR.keys(pattern='mm:topic:*')
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 936, in keys
return self.execute_command('KEYS', pattern)
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/client.py", line 578, in execute_command
connection.send_command(*args)
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 563, in send_command
self.send_packed_command(self.pack_command(*args))
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 538, in send_packed_command
self.connect()
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/redis/connection.py", line 442, in connect
raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 2 connecting to unix socket: /var/run/redis/redis.sock. No such file or directory.
2019-07-06T20:22:28 (1191)launcher.main INFO: Starting mm-run.py version 0.9.62
2019-07-06T20:22:28 (1191)launcher.main INFO: mm-run.py arguments: Namespace(config='/opt/minemeld/local/config', multiprocessing=0, nodes_per_chassis=15.0, verbose=False)
2019-07-06T20:22:30 (1191)config._load_config_from_dir INFO: Switching to candidate config
2019-07-06T20:22:30 (1191)config._load_config_from_dir INFO: Changes in config: []
2019-07-06T20:22:30 (1191)config._destroy_old_nodes INFO: Destroyed nodes: []
2019-07-06T20:22:30 (1191)launcher.main INFO: mm-run.py config: _Config(nodes={'spamhaus_EDROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.EDROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/edrop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'dshield_blocklist': {'output': True, 'config': {'indicator': {'regex': '^([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})\\t([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})', 'transform': '\\1-\\2'}, 'source_name': 'dshield.block', 'age_out': {'default': None, 'sudden_death': True, 'interval': 257}, 'url': 'https://www.dshield.org/block.txt', 'fields': {'dshield_name': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t([^\\t]+)', 'transform': '\\1'}, 'dshield_country': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t([A-Z]+)', 'transform': '\\1'}, 'dshield_nattacks': {'regex': '^.*\\t.*\\t[0-9]+\\t([0-9]+)', 'transform': '\\1'}, 'dshield_email': {'regex': '^.*\\t.*\\t[0-9]+\\t[0-9]+\\t[^\\t]+\\t[A-Z]+\\t(\\S+)', 'transform': '\\1'}}, 'interval': 619, 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '[#S].*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'inboundaggregator': {'inputs': ['spamhaus_DROP', 'spamhaus_EDROP', 'dshield_blocklist', 'wlWhiteListIPv4'], 'config': {'whitelist_prefixes': ['wl'], 'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", "direction == 'inbound'"], 'name': 'accept inbound IPv4', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'", 'direction == null'], 'name': 'accept generic IPv4', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.ipop.AggregateIPv4FT', 'output': True}, 'inboundfeedhc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence > 75', "share_level == 'green'"], 'name': 'accept confidence > 75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'spamhaus_DROP': {'output': True, 'config': {'indicator': {'regex': '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}'}, 'source_name': 'spamhaus.DROP', 'age_out': {'default': None, 'sudden_death': True, 'interval': 677}, 'url': 'https://www.spamhaus.org/drop/drop.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 100, 'share_level': 'green'}, 'ignore_regex': '^;.*'}, 'class': 'minemeld.ft.http.HttpFT'}, 'wlWhiteListIPv4': {'inputs': [], 'config': {'attributes': {'confidence': 100, 'share_level': 'red'}, 'interval': 3600, 'age_out': {'default': None, 'sudden_death': True, 'interval': 67}}, 'class': 'minemeld.ft.local.YamlIPv4FT', 'output': True}, 'inboundfeedlc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence < 50', "share_level == 'green'"], 'name': 'accept confidence < 50 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}, 'inboundfeedmc': {'inputs': ['inboundaggregator'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence >= 50', 'confidence < 75', "share_level == 'green'"], 'name': 'accept confidence 50-75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}}, fabric={'config': {'priority': -2, 'num_connections': 50}, 'class': 'ZMQRedis'}, mgmtbus={'slave': {}, 'master': {}, 'transport': {'config': {'priority': 2, 'num_connections': 10}, 'class': 'ZMQRedis'}}, changes=[])
2019-07-06T20:22:30 (1191)launcher.main INFO: multiprocessing: #cores: 2
2019-07-06T20:22:30 (1191)launcher.main INFO: multiprocessing: max #chassis: 2
2019-07-06T20:22:30 (1191)launcher.main INFO: Number of chassis: 1
2019-07-06T20:22:30 (1203)loader.load INFO: Loading minemeld_nodes:minemeld.ft.http.HttpFT
2019-07-06T20:22:30 (1203)base.read_checkpoint ERROR: spamhaus_EDROP - Error reading last checkpoint
Traceback (most recent call last):
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/ft/base.py", line 255, in read_checkpoint
with open(self.name+'.chkp', 'r') as f:
IOError: [Errno 2] No such file or directory: 'spamhaus_EDROP.chkp'
2019-07-06T20:22:30 (1203)base.state INFO: spamhaus_EDROP - transitioning to state 1
2019-07-06T20:22:30 (1203)loader.load INFO: Loading minemeld_nodes:minemeld.ft.http.HttpFT
2019-07-06T20:22:30 (1203)base.read_checkpoint ERROR: dshield_blocklist - Error reading last checkpoint
Traceback (most recent call last):
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/ft/base.py", line 255, in read_checkpoint
with open(self.name+'.chkp', 'r') as f:
IOError: [Errno 2] No such file or directory: 'dshield_blocklist.chkp'
2019-07-06T20:22:30 (1203)base.state INFO: dshield_blocklist - transitioning to state 1
2019-07-06T20:22:30 (1203)loader.load INFO: Loading minemeld_nodes:minemeld.ft.redis.RedisSet
2019-07-06T20:22:30 (1203)redis.read_checkpoint ERROR: inboundfeedlc - Error reading last checkpoint
Traceback (most recent call last):
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/ft/redis.py", line 69, in read_checkpoint
raise ValueError('{} - last checkpoint not found'.format(self.name))
ValueError: inboundfeedlc - last checkpoint not found
2019-07-06T20:22:30 (1203)base.connect INFO: inboundfeedlc - requesting fabric sub channel for inboundaggregator
2019-07-06T20:22:30 (1203)base.state INFO: inboundfeedlc - transitioning to state 1
2019-07-06T20:22:30 (1203)loader.load INFO: Loading minemeld_nodes:minemeld.ft.redis.RedisSet
2019-07-06T20:22:30 (1203)redis.read_checkpoint ERROR: inboundfeedhc - Error reading last checkpoint
Traceback (most recent call last):
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/ft/redis.py", line 69, in read_checkpoint
raise ValueError('{} - last checkpoint not found'.format(self.name))
ValueError: inboundfeedhc - last checkpoint not found
2019-07-06T20:22:30 (1203)base.connect INFO: inboundfeedhc - requesting fabric sub channel for inboundaggregator
2019-07-06T20:22:30 (1203)base.state INFO: inboundfeedhc - transitioning to state 1
2019-07-06T20:22:30 (1203)loader.load INFO: Loading minemeld_nodes:minemeld.ft.http.HttpFT
2019-07-06T20:22:30 (1203)base.read_checkpoint ERROR: spamhaus_DROP - Error reading last checkpoint
Traceback (most recent call last):
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/ft/base.py", line 255, in read_checkpoint
with open(self.name+'.chkp', 'r') as f:
IOError: [Errno 2] No such file or directory: 'spamhaus_DROP.chkp'
2019-07-06T20:22:30 (1203)base.state INFO: spamhaus_DROP - transitioning to state 1
2019-07-06T20:22:30 (1203)loader.load INFO: Loading minemeld_nodes:minemeld.ft.local.YamlIPv4FT
2019-07-06T20:22:30 (1203)base.read_checkpoint ERROR: wlWhiteListIPv4 - Error reading last checkpoint
Traceback (most recent call last):
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/ft/base.py", line 255, in read_checkpoint
with open(self.name+'.chkp', 'r') as f:
IOError: [Errno 2] No such file or directory: 'wlWhiteListIPv4.chkp'
2019-07-06T20:22:30 (1203)base.state INFO: wlWhiteListIPv4 - transitioning to state 1
2019-07-06T20:22:30 (1203)loader.load INFO: Loading minemeld_nodes:minemeld.ft.ipop.AggregateIPv4FT
2019-07-06T20:22:30 (1203)base.read_checkpoint ERROR: inboundaggregator - Error reading last checkpoint
Traceback (most recent call last):
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/ft/base.py", line 255, in read_checkpoint
with open(self.name+'.chkp', 'r') as f:
IOError: [Errno 2] No such file or directory: 'inboundaggregator.chkp'
2019-07-06T20:22:30 (1203)base.connect INFO: inboundaggregator - requesting fabric sub channel for spamhaus_DROP
2019-07-06T20:22:30 (1203)base.connect INFO: inboundaggregator - requesting fabric sub channel for spamhaus_EDROP
2019-07-06T20:22:30 (1203)base.connect INFO: inboundaggregator - requesting fabric sub channel for dshield_blocklist
2019-07-06T20:22:30 (1203)base.connect INFO: inboundaggregator - requesting fabric sub channel for wlWhiteListIPv4
2019-07-06T20:22:30 (1203)base.state INFO: inboundaggregator - transitioning to state 1
2019-07-06T20:22:30 (1203)loader.load INFO: Loading minemeld_nodes:minemeld.ft.redis.RedisSet
2019-07-06T20:22:30 (1203)redis.read_checkpoint ERROR: inboundfeedmc - Error reading last checkpoint
Traceback (most recent call last):
File "/opt/minemeld/engine/0.9.62/local/lib/python2.7/site-packages/minemeld/ft/redis.py", line 69, in read_checkpoint
raise ValueError('{} - last checkpoint not found'.format(self.name))
ValueError: inboundfeedmc - last checkpoint not found
2019-07-06T20:22:30 (1203)base.connect INFO: inboundfeedmc - requesting fabric sub channel for inboundaggregator
2019-07-06T20:22:30 (1203)base.state INFO: inboundfeedmc - transitioning to state 1
2019-07-06T20:22:30 (1191)mgmtbus.init_graph INFO: state: {u'mbus:slave:wlWhiteListIPv4': {u'checkpoint': None, u'is_source': True, u'state': 1}, u'mbus:slave:spamhaus_DROP': {u'checkpoint': None, u'is_source': True, u'state': 1}, u'mbus:slave:inboundfeedhc': {u'checkpoint': None, u'is_source': False, u'state': 1}, u'mbus:slave:inboundaggregator': {u'checkpoint': None, u'is_source': False, u'state': 1}, u'mbus:slave:dshield_blocklist': {u'checkpoint': None, u'is_source': True, u'state': 1}, u'mbus:slave:inboundfeedlc': {u'checkpoint': None, u'is_source': False, u'state': 1}, u'mbus:slave:inboundfeedmc': {u'checkpoint': None, u'is_source': False, u'state': 1}, u'mbus:slave:spamhaus_EDROP': {u'checkpoint': None, u'is_source': True, u'state': 1}}
2019-07-06T20:22:30 (1191)mgmtbus.init_graph INFO: changes: []
2019-07-06T20:22:30 (1191)startupplanner._plan_subgraph INFO: state_info: {u'spamhaus_EDROP': {u'checkpoint': None, u'is_source': True, u'state': 1}, u'dshield_blocklist': {u'checkpoint': None, u'is_source': True, u'state': 1}, u'inboundfeedlc': {u'checkpoint': None, u'is_source': False, u'state': 1}, u'inboundfeedhc': {u'checkpoint': None, u'is_source': False, u'state': 1}, u'spamhaus_DROP': {u'checkpoint': None, u'is_source': True, u'state': 1}, u'wlWhiteListIPv4': {u'checkpoint': None, u'is_source': True, u'state': 1}, u'inboundaggregator': {u'checkpoint': None, u'is_source': False, u'state': 1}, u'inboundfeedmc': {u'checkpoint': None, u'is_source': False, u'state': 1}}
2019-07-06T20:22:30 (1191)startupplanner._plan_subgraph INFO: planning for subgraph ['spamhaus_EDROP', 'dshield_blocklist', 'inboundfeedlc', 'inboundfeedhc', 'spamhaus_DROP', 'wlWhiteListIPv4', 'inboundaggregator', 'inboundfeedmc']
2019-07-06T20:22:30 (1191)startupplanner._plan_subgraph INFO: No checkpoints, new graph: reset
2019-07-06T20:22:30 (1191)mgmtbus.init_graph INFO: spamhaus_EDROP <= reset
2019-07-06T20:22:30 (1203)base.state INFO: spamhaus_EDROP - transitioning to state 3
2019-07-06T20:22:31 (1203)base.state INFO: spamhaus_EDROP - transitioning to state 4
2019-07-06T20:22:31 (1191)mgmtbus.init_graph INFO: dshield_blocklist <= reset
2019-07-06T20:22:31 (1203)base.state INFO: dshield_blocklist - transitioning to state 3
2019-07-06T20:22:31 (1203)base.state INFO: dshield_blocklist - transitioning to state 4
2019-07-06T20:22:31 (1191)mgmtbus.init_graph INFO: inboundfeedlc <= reset
2019-07-06T20:22:31 (1203)base.state INFO: inboundfeedlc - transitioning to state 3
2019-07-06T20:22:31 (1203)base.state INFO: inboundfeedlc - transitioning to state 4
2019-07-06T20:22:31 (1191)mgmtbus.init_graph INFO: inboundfeedhc <= reset
2019-07-06T20:22:31 (1203)base.state INFO: inboundfeedhc - transitioning to state 3
2019-07-06T20:22:31 (1203)base.state INFO: inboundfeedhc - transitioning to state 4
2019-07-06T20:22:31 (1191)mgmtbus.init_graph INFO: spamhaus_DROP <= reset
2019-07-06T20:22:31 (1203)base.state INFO: spamhaus_DROP - transitioning to state 3
2019-07-06T20:22:31 (1203)base.state INFO: spamhaus_DROP - transitioning to state 4
2019-07-06T20:22:31 (1191)mgmtbus.init_graph INFO: wlWhiteListIPv4 <= reset
2019-07-06T20:22:31 (1203)base.state INFO: wlWhiteListIPv4 - transitioning to state 3
2019-07-06T20:22:31 (1203)base.state INFO: wlWhiteListIPv4 - transitioning to state 4
2019-07-06T20:22:31 (1191)mgmtbus.init_graph INFO: inboundaggregator <= reset
2019-07-06T20:22:31 (1203)base.state INFO: inboundaggregator - transitioning to state 3
2019-07-06T20:22:31 (1203)base.state INFO: inboundaggregator - transitioning to state 4
2019-07-06T20:22:31 (1191)mgmtbus.init_graph INFO: inboundfeedmc <= reset
2019-07-06T20:22:31 (1203)base.state INFO: inboundfeedmc - transitioning to state 3
2019-07-06T20:22:31 (1203)base.state INFO: inboundfeedmc - transitioning to state 4
2019-07-06T20:22:31 (1203)chassis.mgmtbus_start INFO: chassis - start received from mgmtbus
2019-07-06T20:22:31 (1203)chassis.start INFO: chassis start called
2019-07-06T20:22:31 (1203)base.state INFO: spamhaus_EDROP - transitioning to state 5
2019-07-06T20:22:31 (1203)base.state INFO: dshield_blocklist - transitioning to state 5
2019-07-06T20:22:31 (1203)base.state INFO: inboundaggregator - transitioning to state 5
2019-07-06T20:22:31 (1203)base.state INFO: inboundfeedhc - transitioning to state 5
2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_EDROP - command: 1562440951063 age_out
2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _age_out: 0
2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: dshield_blocklist - command: 1562440951064 age_out
2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _age_out: 0
2019-07-06T20:22:31 (1203)basepoller._huppable_wait INFO: hup is clear: False
2019-07-06T20:22:31 (1203)base.state INFO: spamhaus_DROP - transitioning to state 5
2019-07-06T20:22:31 (1203)base.state INFO: wlWhiteListIPv4 - transitioning to state 5
2019-07-06T20:22:31 (1203)base.state INFO: inboundfeedlc - transitioning to state 5
2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_EDROP - command: 1562440951065 poll
2019-07-06T20:22:31 (1203)basepoller._polling_loop INFO: Polling spamhaus_EDROP
2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_DROP - command: 1562440951077 age_out
2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _age_out: 0
2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: wlWhiteListIPv4 - command: 1562440951077 age_out
2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _age_out: 0
2019-07-06T20:22:31 (1203)basepoller._huppable_wait INFO: hup is clear: False
2019-07-06T20:22:31 (1203)base.state INFO: inboundfeedmc - transitioning to state 5
2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_DROP - command: 1562440951078 poll
2019-07-06T20:22:31 (1203)basepoller._polling_loop INFO: Polling spamhaus_DROP
2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_EDROP - command: 1562440951065 sudden_death
2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _last_run: 0
2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_EDROP - command: 1562440951065 age_out
2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _age_out: 0
2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_EDROP - command: 1562440951065 gc
2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _withdrawn: 0
2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_DROP - command: 1562440951078 sudden_death
2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _last_run: 0
2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_DROP - command: 1562440951078 age_out
2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _age_out: 0
2019-07-06T20:22:31 (1203)basepoller._actor_loop INFO: spamhaus_DROP - command: 1562440951078 gc
2019-07-06T20:22:31 (1203)table._query_by_index INFO: Deleted in scan of _withdrawn: 0
2019-07-06T20:22:32 (1203)launcher._run_chassis INFO: Nodes initialized
2019-07-06T20:22:33 (1203)basepoller._huppable_wait INFO: hup is clear: False
2019-07-06T20:22:33 (1203)basepoller._huppable_wait INFO: hup is clear: False
2019-07-06T20:22:33 (1203)basepoller._actor_loop INFO: dshield_blocklist - command: 1562440953183 poll
2019-07-06T20:22:33 (1203)basepoller._polling_loop INFO: Polling dshield_blocklist
2019-07-06T20:22:33 (1203)basepoller._actor_loop INFO: wlWhiteListIPv4 - command: 1562440953183 poll
2019-07-06T20:22:33 (1203)basepoller._polling_loop INFO: Polling wlWhiteListIPv4
2019-07-06T20:22:33 (1203)basepoller._actor_loop INFO: wlWhiteListIPv4 - command: 1562440953183 sudden_death
2019-07-06T20:22:33 (1203)basepoller._actor_loop INFO: wlWhiteListIPv4 - command: 1562440953183 age_out
2019-07-06T20:22:33 (1203)table._query_by_index INFO: Deleted in scan of _age_out: 0
2019-07-06T20:22:33 (1203)basepoller._actor_loop INFO: wlWhiteListIPv4 - command: 1562440953183 gc
2019-07-06T20:22:33 (1203)table._query_by_index INFO: Deleted in scan of _withdrawn: 0
2019-07-06T20:22:34 (1203)basepoller._actor_loop INFO: dshield_blocklist - command: 1562440953183 sudden_death
2019-07-06T20:22:34 (1203)table._query_by_index INFO: Deleted in scan of _last_run: 0
2019-07-06T20:22:34 (1203)basepoller._actor_loop INFO: dshield_blocklist - command: 1562440953183 age_out
2019-07-06T20:22:34 (1203)table._query_by_index INFO: Deleted in scan of _age_out: 0
2019-07-06T20:22:34 (1203)basepoller._actor_loop INFO: dshield_blocklist - command: 1562440953183 gc
2019-07-06T20:22:34 (1203)table._query_by_index INFO: Deleted in scan of _withdrawn: 0
... View more
12-08-2018
01:33 PM
1 Kudo
Hi guys, looks like there is an issue with the URL processor. Please could you check this out as it breaks OneNote.
The original input from Microsoft includes for instance two URLs "cdn.onenote.net" and "site-cdn.onenote.net" under "id": 271, and these two are aggregated to "*cdn.onenote.net" which is invalid as the wildcard and charaters cannot be in the same token.
In total there are 11 entires like this and the FireWall log shows that it skipped exactly 11
"Office365-URL, 1, 1 url) Valid entries(578) lines skipped(11)"
*broadcast.officeapps.live.com
*cdn.onenote.net
*excel.officeapps.live.com
*onenote.officeapps.live.com
*powerpoint.officeapps.live.com
*view.officeapps.live.com
*visio.officeapps.live.com
*word-edit.officeapps.live.com
*word-view.officeapps.live.com
*-files.sharepoint.com
*-myfiles.sharepoint.com
... View more
04-17-2018
03:09 PM
I just tried to deploy MineMeld in Azure using the official template and got the same error.
admin@vm1:~$ sudo -u minemeld /opt/minemeld/engine/current/bin/supervisorctl -c /opt/minemeld/local/supervisor/config/supervisord.conf status
minemeld-engine FATAL Exited too quickly (process log may have details)
minemeld-supervisord-listener RUNNING pid 11603, uptime 0:01:29
minemeld-traced RUNNING pid 11605, uptime 0:01:29
minemeld-web RUNNING pid 12267, uptime 0:00:01
admin@vm1:~$ less /opt/minemeld/log/
minemeld-engine.log minemeld-supervisord-listener-stdout---supervisor-y5ajV9.log minemeld-web.log
minemeld-engine-stderr---supervisor-M96AeT.log minemeld-traced.log minemeld-web-stderr---supervisor-fsqwLy.log
minemeld-supervisord-listener.log minemeld-traced-stderr---supervisor-_qh1ic.log supervisord.log
admin@vm1:~$ less /opt/minemeld/log/minemeld-engine.log
2018-04-17T22:01:59 (11604)launcher.main INFO: Starting mm-run.py version 0.9.44
2018-04-17T22:01:59 (11604)launcher.main INFO: mm-run.py arguments: Namespace(config='/opt/minemeld/local/config', multiprocessing=0, nodes_per_chassis=15.0, verbose=False)
Traceback (most recent call last):
File "/opt/minemeld/engine/current/bin/mm-run", line 11, in <module>
sys.exit(main())
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 218, in main
config = minemeld.run.config.load_config(args.config)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/config.py", line 567, in load_config
return _load_config_from_dir(config_path)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/config.py", line 382, in _load_config_from_dir
ccvalid, cconfig = _load_and_validate_config_from_file(ccpath)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/config.py", line 281, in _load_and_validate_config_from_file
valid = resolve_prototypes(config)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/config.py", line 477, in resolve_prototypes
prototypes_entrypoints = minemeld.loader.map(minemeld.loader.MM_PROTOTYPES_ENTRYPOINT)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/loader.py", line 105, in map
_initialize_entry_point_group(entrypoint_group)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/loader.py", line 55, in _initialize_entry_point_group
installed = _installed_versions()
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/loader.py", line 29, in _installed_versions
installed_dists = pip.get_installed_distributions()
AttributeError: 'module' object has no attribute 'get_installed_distributions'
2018-04-17T22:02:02 (11748)launcher.main INFO: Starting mm-run.py version 0.9.44
2018-04-17T22:02:02 (11748)launcher.main INFO: mm-run.py arguments: Namespace(config='/opt/minemeld/local/config', multiprocessing=0, nodes_per_chassis=15.0, verbose=False)
Traceback (most recent call last):
File "/opt/minemeld/engine/current/bin/mm-run", line 11, in <module>
sys.exit(main())
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 218, in main
config = minemeld.run.config.load_config(args.config)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/config.py", line 567, in load_config
return _load_config_from_dir(config_path)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/config.py", line 382, in _load_config_from_dir
ccvalid, cconfig = _load_and_validate_config_from_file(ccpath)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/config.py", line 281, in _load_and_validate_config_from_file
valid = resolve_prototypes(config)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/config.py", line 477, in resolve_prototypes
prototypes_entrypoints = minemeld.loader.map(minemeld.loader.MM_PROTOTYPES_ENTRYPOINT)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/loader.py", line 105, in map
_initialize_entry_point_group(entrypoint_group)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/loader.py", line 55, in _initialize_entry_point_group
installed = _installed_versions()
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/loader.py", line 29, in _installed_versions
installed_dists = pip.get_installed_distributions()
AttributeError: 'module' object has no attribute 'get_installed_distributions'
2018-04-17T22:02:06 (11770)launcher.main INFO: Starting mm-run.py version 0.9.44
2018-04-17T22:02:06 (11770)launcher.main INFO: mm-run.py arguments: Namespace(config='/opt/minemeld/local/config', multiprocessing=0, nodes_per_chassis=15.0, verbose=False)
Traceback (most recent call last):
File "/opt/minemeld/engine/current/bin/mm-run", line 11, in <module>
sys.exit(main())
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 218, in main
config = minemeld.run.config.load_config(args.config)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/config.py", line 567, in load_config
return _load_config_from_dir(config_path)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/config.py", line 382, in _load_config_from_dir
ccvalid, cconfig = _load_and_validate_config_from_file(ccpath)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/config.py", line 281, in _load_and_validate_config_from_file
valid = resolve_prototypes(config)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/config.py", line 477, in resolve_prototypes
prototypes_entrypoints = minemeld.loader.map(minemeld.loader.MM_PROTOTYPES_ENTRYPOINT)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/loader.py", line 105, in map
_initialize_entry_point_group(entrypoint_group)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/loader.py", line 55, in _initialize_entry_point_group
installed = _installed_versions()
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/loader.py", line 29, in _installed_versions
installed_dists = pip.get_installed_distributions()
AttributeError: 'module' object has no attribute 'get_installed_distributions'
2018-04-17T22:02:10 (11798)launcher.main INFO: Starting mm-run.py version 0.9.44
2018-04-17T22:02:10 (11798)launcher.main INFO: mm-run.py arguments: Namespace(config='/opt/minemeld/local/config', multiprocessing=0, nodes_per_chassis=15.0, verbose=False)
Traceback (most recent call last):
File "/opt/minemeld/engine/current/bin/mm-run", line 11, in <module>
sys.exit(main())
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 218, in main
config = minemeld.run.config.load_config(args.config)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/config.py", line 567, in load_config
return _load_config_from_dir(config_path)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/config.py", line 382, in _load_config_from_dir
ccvalid, cconfig = _load_and_validate_config_from_file(ccpath)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/config.py", line 281, in _load_and_validate_config_from_file
valid = resolve_prototypes(config)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/run/config.py", line 477, in resolve_prototypes
prototypes_entrypoints = minemeld.loader.map(minemeld.loader.MM_PROTOTYPES_ENTRYPOINT)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/loader.py", line 105, in map
_initialize_entry_point_group(entrypoint_group)
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/loader.py", line 55, in _initialize_entry_point_group
installed = _installed_versions()
File "/opt/minemeld/engine/0.9.44/local/lib/python2.7/site-packages/minemeld/loader.py", line 29, in _installed_versions
installed_dists = pip.get_installed_distributions()
AttributeError: 'module' object has no attribute 'get_installed_distributions'
... View more
04-05-2017
05:55 AM
Does anyone know what the "IP Tag" log is exactly used for? I know its related to dynamic address objects but what is the log used for?
... View more
LEGAL NOTICES
Copyright 2007 - 2023 - Palo Alto Networks