topic Re: How do I create my list of blocked IPs for firewall to feed from ? in General Topics

How do I create my list of blocked IPs for firewall to feed from ?

niuk — Thu, 24 Nov 2016 20:25:49 GMT

I need to create my list 'MineMeld-source-List' of blocked IPs which I want to use in the rule. I tried to use prototype stdlib.listIPv4Generic as input where I can add indicators. Then used stdlib.aggregatorIPv4Inbound based aggregator and subsribed firewall to stdlib.feedHCGreen based output (MineMeld-source-List). But on firewall I am getting warning EDL(vsys1/MineMeld-source-List ip) Downloaded file is either not a text file or empty file during policy commit. In the Logs/System I can see 'EDL(MineMeld-source-List) EDL Fetch job done' every 5 min but it is not working. Also on firewall I can see:

admin@MR-DC(active)> request system external-list show type ip name MineMeld-source-List
 
Server error : entry not found

Re: How do I create my list of blocked IPs for firewall to feed from ?

lmori — Thu, 24 Nov 2016 22:40:44 GMT

Hi @niuk,

please could you share you MineMeld config ? You can export it from the CONFIG tab.

Thanks !
luigi

Re: How do I create my list of blocked IPs for firewall to feed from ?

niuk — Fri, 25 Nov 2016 01:15:56 GMT

Here it is, I am referring to 'path' with 'source-*', so source-input, source-agggregator and source-output

nodes:
  spamhaus_EDROP:
    output: true
    prototype: spamhaus.EDROP
  dshield_blocklist:
    output: true
    prototype: dshield.block
  inboundaggregator:
    inputs:
      - spamhaus_DROP
      - spamhaus_EDROP
      - dshield_blocklist
      - wlWhiteListIPv4
      - panos_syslog_miner
    output: true
    prototype: stdlib.aggregatorIPv4Inbound
  inboundfeedhc:
    inputs:
      - inboundaggregator
    output: false
    prototype: stdlib.feedHCGreen
  spamhaus_DROP:
    output: true
    prototype: spamhaus.DROP
  wlWhiteListIPv4:
    inputs: []
    output: true
    prototype: stdlib.listIPv4Generic
  inboundfeedlc:
    inputs:
      - inboundaggregator
    output: false
    prototype: stdlib.feedLCGreen
  inboundfeedmc:
    inputs:
      - inboundaggregator
    output: false
    prototype: stdlib.feedMCGreen
  panos_syslog_miner:
    inputs: []
    output: true
    prototype: stdlib.syslogMiner
  syslog_analyzer:
    inputs:
      - inboundaggregator
    output: true
    prototype: stdlib.localSyslog
  source-WhiteList:
    inputs: []
    output: true
    prototype: stdlib.listIPv4Generic
  source-aggregator:
    inputs:
      - source-WhiteList
    output: true
    prototype: stdlib.aggregatorIPv4Inbound
  source-output:
    inputs:
      - source-aggregator
    output: false
    prototype: stdlib.feedHCGreen

Re: How do I create my list of blocked IPs for firewall to feed from ?

lmori — Fri, 25 Nov 2016 07:21:18 GMT

Hi @niuk !

You have selected feedHCGreen, this output accepts only indicator with confidence above 75 (and by default indicators created in listIPv4Generic have confidence 100) and with share level green. Please double check all the indicators you have created are Green. Also the aggregator inbound accepts only indicator with direction Inbound, once again please check the indicators you have created have direction INBOUND or UNKNOWN.

Once done you should be able to access your feed at https://<minemeld ip address>/feeds/source-output

luigi

Re: How do I create my list of blocked IPs for firewall to feed from ?

niuk — Sat, 26 Nov 2016 12:22:30 GMT

It works now after changiung direction and share level. 'request system external..' still shows server error, but I can see the ip addresses dropped in logs by the rule using my MineMeld-source-List

admin@MR-DC1-PFWP02(active)> request system external-list show type ip name MineMeld-source-List 

Server error : entry not found

Re: How do I create my list of blocked IPs for firewall to feed from ?

niuk — Sat, 26 Nov 2016 13:40:59 GMT

One more thing, I updated my MineMeld-source-List but on firewall I can see that 'EDL(MineMeld-source-List) No changes to list file' ? And it is not working for updated IP (I reloaded indicator list)

Re: How do I create my list of blocked IPs for firewall to feed from ?

lmori — Sun, 27 Nov 2016 20:50:53 GMT

Hi @niuk,

- check with the browser going directly to "https://<minemeld ip address>/feeds/source-output", do you see all the indicators you have creted ? If not:

- check inside the MineMeld logs with the following query: "source:source-output op:DROP_UPDATE" to see if some indicators have been dropped by the feed

- check if the EDL object is point to the right URL (https://<minemeld ip address>/feeds/source-output)

- check inside the ms.log on PAN-OS for errors around EDL download

Luigi

Re: How do I create my list of blocked IPs for firewall to feed from ?

niuk — Sun, 27 Nov 2016 21:24:28 GMT

- "https://<minemeld ip address>/feeds/source-output" is showing all the indicators I creted

10.199.107.10-10.199.107.10
192.168.3.0-192.168.3.255

- nothing in "source:source-output op:DROP_UPDATE" but .. logs don't go too far because I receiverd Error receiving outputs Metrics internal error and restarted server

- the EDL object points to the right URL I can test it with button click and as I said it is working fine for

192.168.3.0-192.168.3.255

but not for which was added later, after feed created

10.199.107.10-10.199.107.10

But I 've noticed that after restarting MineMeld I have all Indicatiors blocked correctly by firewall. It happened to me that I had to restart server second time, practically every 2 days (I've got this internal error second time).

Re: How do I create my list of blocked IPs for firewall to feed from ?

lmori — Sun, 27 Nov 2016 21:30:30 GMT

Hi @niuk,

logs are stored on disk, you don't lose them with restarts.

Could you send me your /opt/minemeld/log/minemeld-engine.log and /opt/minemeld/log/minemeld-web.log files in a zip at lmori@paloaltonetworks.com ? I'd like to give a look at the internal errors.

Thanks,

luigi

Re: How do I create my list of blocked IPs for firewall to feed from ?

Fengrui — Fri, 09 Dec 2016 16:38:13 GMT

The error message

Server error : entry not found

is most likely caused by not setting the vsys, if you do,

> set system setting target-vsys vsys1

This should work.