DShield list

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Please sign in to see details of an important advisory in our Customer Advisories area.

DShield list

L4 Transporter

Hi guys,

 

recently I've noticed a strange behaviour in my DShield miner (dshield.block prototype, https://www.dshield.org/block.txt). I have 2 MM (0.9.46 and 0.9.48), and both present the problem. Time to time, I can't precise the period, in fact it is not regular, my miner presents 0 indicators mined.

 

First, I would like to know if somebody else experienced it. Second, how you dealed with it?

 

Thank you in advanced.

Best regards.

1 accepted solution

Accepted Solutions

Hi guys,

 

pretty sure the problem was in DShield side. I didn't change anything in my MM Machine or configuration and the behaviour returned to normal ( the same way it began its erroneous behaviour). Suddenly.

 

I'm closing this topic.

 

Best regards.

View solution in original post

15 REPLIES 15

L7 Applicator

Hi @danilo.souza,

could you check the logs? Did the Miner polled 0 valid indicatos from the feed?

Hi @lmori

 

Look the images attached. In the first one my panel shows 0 indicators to dshield miner. The second one, shows the log. At 10hs it withdraws the indicators but don't emmit the new ones. It stays almost 30 min with 0 indicators.

 

Any idea.

 

Thank you.

Hi @danilo.souza,

my fault, I meant the engine logs (System > Engine > Logs). Could you check for errors on the dshield miner?

Hi @lmori

 

this is what I got, recently, related to Dshield miner.

 

2018-08-14T11:16:47 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256207149 poll
2018-08-14T11:16:47 (31433)basepoller._polling_loop INFO: Polling dshield_blocklist
2018-08-14T11:16:47 (31433)connectionpool._new_conn INFO: Starting new HTTPS connection (1): www.dshield.org

.
.
.

2018-08-14T11:16:56 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256207149 sudden_death
2018-08-14T11:16:56 (31433)table._query_by_index INFO: Deleted in scan of _last_run: 20
2018-08-14T11:16:56 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256207149 age_out
2018-08-14T11:16:56 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0
2018-08-14T11:16:56 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256207149 gc
2018-08-14T11:16:56 (31433)table._query_by_index INFO: Deleted in scan of _withdrawn: 0
2018-08-14T11:17:42 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256262787 age_out
2018-08-14T11:17:42 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0

.
.
.

2018-08-14T11:21:59 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256519854 age_out
2018-08-14T11:21:59 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0

.
.
.

2018-08-14T11:26:16 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256776863 age_out
2018-08-14T11:26:16 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0

.
.
.

2018-08-14T11:27:06 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256826150 poll
2018-08-14T11:27:06 (31433)basepoller._polling_loop INFO: Polling dshield_blocklist
2018-08-14T11:27:06 (31433)connectionpool._new_conn INFO: Starting new HTTPS connection (1): www.dshield.org
2018-08-14T11:27:06 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256826150 sudden_death
2018-08-14T11:27:06 (31433)table._query_by_index INFO: Deleted in scan of _last_run: 20
2018-08-14T11:27:06 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256826150 age_out
2018-08-14T11:27:06 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0
2018-08-14T11:27:06 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256826150 gc
2018-08-14T11:27:06 (31433)table._query_by_index INFO: Deleted in scan of _withdrawn: 0

.
.
.

2018-08-14T11:30:33 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534257033869 age_out
2018-08-14T11:30:33 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0
2018-08-14T11:30:58 (31432)basepoller._huppable_wait INFO: hup is clear: False
2018-08-14T11:30:58 (31432)basepoller._huppable_wait INFO: hup is clear: False

.
.
.

2018-08-14T11:34:50 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534257290921 age_out
2018-08-14T11:34:50 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0

 

 

 

You can see the "Deleted in scan of _last_run: 20".

 

I could not put here the whole log file, too big. It helps?

 

Thank you.

 

Hi @lmori

 

I put a respost here, but it was calassified as "Spam"! How to deal with it? Was it because I wrote down some log lines?

 

Thank you.

Hi @lmori

 

this is what I got, recently, related to Dshield miner.

 

2018-08-14T11:16:47 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256207149 poll
2018-08-14T11:16:47 (31433)basepoller._polling_loop INFO: Polling dshield_blocklist
2018-08-14T11:16:47 (31433)connectionpool._new_conn INFO: Starting new HTTPS connection (1): www.dshield.org

.
.
.

2018-08-14T11:16:56 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256207149 sudden_death
2018-08-14T11:16:56 (31433)table._query_by_index INFO: Deleted in scan of _last_run: 20
2018-08-14T11:16:56 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256207149 age_out
2018-08-14T11:16:56 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0
2018-08-14T11:16:56 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256207149 gc
2018-08-14T11:16:56 (31433)table._query_by_index INFO: Deleted in scan of _withdrawn: 0
2018-08-14T11:17:42 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256262787 age_out
2018-08-14T11:17:42 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0

.
.
.

2018-08-14T11:21:59 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256519854 age_out
2018-08-14T11:21:59 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0

.
.
.

2018-08-14T11:26:16 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256776863 age_out
2018-08-14T11:26:16 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0

.
.
.

2018-08-14T11:27:06 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256826150 poll
2018-08-14T11:27:06 (31433)basepoller._polling_loop INFO: Polling dshield_blocklist
2018-08-14T11:27:06 (31433)connectionpool._new_conn INFO: Starting new HTTPS connection (1): www.dshield.org
2018-08-14T11:27:06 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256826150 sudden_death
2018-08-14T11:27:06 (31433)table._query_by_index INFO: Deleted in scan of _last_run: 20
2018-08-14T11:27:06 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256826150 age_out
2018-08-14T11:27:06 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0
2018-08-14T11:27:06 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256826150 gc
2018-08-14T11:27:06 (31433)table._query_by_index INFO: Deleted in scan of _withdrawn: 0

.
.
.

2018-08-14T11:30:33 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534257033869 age_out
2018-08-14T11:30:33 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0
2018-08-14T11:30:58 (31432)basepoller._huppable_wait INFO: hup is clear: False
2018-08-14T11:30:58 (31432)basepoller._huppable_wait INFO: hup is clear: False

.
.
.

2018-08-14T11:34:50 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534257290921 age_out
2018-08-14T11:34:50 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0

 

 

 

You can see the "Deleted in scan of _last_run: 20".

 

I could not put here the whole log file, too big. It helps?

 

Thank you.

Hi @danilo.souza,

never happened to me before, sorry about that. Are you on the Slack community? Could you PM the logs there?

 

Thanks,

luigi

Hi @lmori

 

I'm not. Let me try again.

 

Thank you.

Hi

 

this is what I got, recently, related to Dshield miner.

 

2018-08-14T11:16:47 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256207149 poll
2018-08-14T11:16:47 (31433)basepoller._polling_loop INFO: Polling dshield_blocklist
2018-08-14T11:16:47 (31433)connectionpool._new_conn INFO: Starting new HTTPS connection (1): www.dshield.org

.
.
.

2018-08-14T11:16:56 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256207149 sudden_death
2018-08-14T11:16:56 (31433)table._query_by_index INFO: Deleted in scan of _last_run: 20
2018-08-14T11:16:56 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256207149 age_out
2018-08-14T11:16:56 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0
2018-08-14T11:16:56 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256207149 gc
2018-08-14T11:16:56 (31433)table._query_by_index INFO: Deleted in scan of _withdrawn: 0
2018-08-14T11:17:42 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256262787 age_out
2018-08-14T11:17:42 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0

.
.
.

2018-08-14T11:21:59 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256519854 age_out
2018-08-14T11:21:59 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0

.
.
.

2018-08-14T11:26:16 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256776863 age_out
2018-08-14T11:26:16 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0

.
.
.

2018-08-14T11:27:06 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256826150 poll
2018-08-14T11:27:06 (31433)basepoller._polling_loop INFO: Polling dshield_blocklist
2018-08-14T11:27:06 (31433)connectionpool._new_conn INFO: Starting new HTTPS connection (1): www.dshield.org
2018-08-14T11:27:06 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256826150 sudden_death
2018-08-14T11:27:06 (31433)table._query_by_index INFO: Deleted in scan of _last_run: 20
2018-08-14T11:27:06 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256826150 age_out
2018-08-14T11:27:06 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0
2018-08-14T11:27:06 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256826150 gc
2018-08-14T11:27:06 (31433)table._query_by_index INFO: Deleted in scan of _withdrawn: 0

.
.
.

2018-08-14T11:30:33 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534257033869 age_out
2018-08-14T11:30:33 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0
2018-08-14T11:30:58 (31432)basepoller._huppable_wait INFO: hup is clear: False
2018-08-14T11:30:58 (31432)basepoller._huppable_wait INFO: hup is clear: False

.
.
.

2018-08-14T11:34:50 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534257290921 age_out
2018-08-14T11:34:50 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0

 

 

 

You can see the "Deleted in scan of _last_run: 20".

 

I could not put here the whole log file, too big. It helps?

 

Thank you.

Hi @lmori

 

this is what I got, recently, related to Dshield miner.


2018-08-14T11:16:47 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256207149 poll
2018-08-14T11:16:47 (31433)basepoller._polling_loop INFO: Polling dshield_blocklist
2018-08-14T11:16:47 (31433)connectionpool._new_conn INFO: Starting new HTTPS connection (1): www.dshield.org

...

2018-08-14T11:16:56 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256207149 sudden_death
2018-08-14T11:16:56 (31433)table._query_by_index INFO: Deleted in scan of _last_run: 20
2018-08-14T11:16:56 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256207149 age_out
2018-08-14T11:16:56 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0
2018-08-14T11:16:56 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256207149 gc
2018-08-14T11:16:56 (31433)table._query_by_index INFO: Deleted in scan of _withdrawn: 0
2018-08-14T11:17:42 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256262787 age_out
2018-08-14T11:17:42 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0

...

2018-08-14T11:21:59 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256519854 age_out
2018-08-14T11:21:59 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0

...

2018-08-14T11:26:16 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256776863 age_out
2018-08-14T11:26:16 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0

...

2018-08-14T11:27:06 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256826150 poll
2018-08-14T11:27:06 (31433)basepoller._polling_loop INFO: Polling dshield_blocklist
2018-08-14T11:27:06 (31433)connectionpool._new_conn INFO: Starting new HTTPS connection (1): www.dshield.org
2018-08-14T11:27:06 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256826150 sudden_death
2018-08-14T11:27:06 (31433)table._query_by_index INFO: Deleted in scan of _last_run: 20
2018-08-14T11:27:06 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256826150 age_out
2018-08-14T11:27:06 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0
2018-08-14T11:27:06 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534256826150 gc
2018-08-14T11:27:06 (31433)table._query_by_index INFO: Deleted in scan of _withdrawn: 0

...

2018-08-14T11:30:33 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534257033869 age_out
2018-08-14T11:30:33 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0
2018-08-14T11:30:58 (31432)basepoller._huppable_wait INFO: hup is clear: False
2018-08-14T11:30:58 (31432)basepoller._huppable_wait INFO: hup is clear: False

...

2018-08-14T11:34:50 (31433)basepoller._actor_loop INFO: dshield_blocklist - command: 1534257290921 age_out
2018-08-14T11:34:50 (31433)table._query_by_index INFO: Deleted in scan of _age_out: 0

 


You can see the "Deleted in scan of _last_run: 20".

 

I could not put here the whole log file, too big. It helps?

 

Thank you.

Hi @lmori

 

I'm attaching a txt. Let me know if you can see it.

 

This is what I got, recently, related to Dshield miner. You can see the "Deleted in scan of _last_run: 20". It helps?

 

Best regards.

Hi guys,

 

Any idea? Any help with this issue? 

 

Thank you.

Best regards.

@danilo.souza,

 

if you're using the default age out policy for the dshield prototype (default: null, interval: 257, sudden_death: true) then the only reason for the node to withdraw all indicators is a successfull poll returning zero indicators.

 

The dshield prototype uses the HttpFT class which parses the HTML page received from https://www.dshield.org/block.txt. A SSL decryption page between the MineMeld instance and Internet might provide a valid HTML page (captive portal) resulting in zero indicators after the parsing stage.

 

Is that a feasible explanation?

Hi @xhoms

 

thank you for the reply. In truth, my MM VM is in a zone that demands/show the Captive Portal when the Firewall is not capable to identify the user. However, I'm not able to asure that this is the problem. I have a very stable environment (don't remember any change in the Authentication Policies) and it was working fine until  few days before I reported it here (sorry, that was when I noticed it, may be the problem was ocurring even earlier). But, if nobody else experimented it, the problem is in my environment.

 

Any news I will share with you. Any help I would appreciate.

 

Best regards.

 

 

  • 1 accepted solution
  • 9796 Views
  • 15 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!