About EdwinD

In an attempt to understand MineMeld better, I have made multiple attempts at setting up a simple DShield configuration.   At some point or another I have had a configuration where indicators age out from either the miner, the aggregator or the output node.   I believe it was when I used the Hail a TAXII miner the indicators eventually age out of the miner itself.   Before I opened this discussion I spent a lot of time in this forum, in the KB articles as well as looking at the discussions at the source repo. I discovered that I want to use the an age_out default of null and sudden_death of true: age_out: default: null sudden_death: true interval: 300   With my currently configuration I have the entire DShield list in my DShield specific miner and aggregator.    For testing, I have a simple html text output node using this aggregator.  The indicators are currently all listed in this output node.  The output is simple IP ranges, formatted like the DShield feed located here: http://panwdbl.appspot.com/   I have MineMeld TAXII output node with this same aggregator as the input.   This TAXII node is feeding into ProofPoint TRAP (Threat Response Auto Pull).   ProofPoint lists 0 indicators in this TAXII feed.    For what it is worth, I have several other MineMeld output nodes which are working.  The difference is that the majority of these originate from MineMeld TAXII miners mining STIX from other TAXII feeds.  Unlike the raw DShield text block list miner, these other miners of STIX data have timestamps which are correctly updating at each poll.         In prior configurations I would see the DShield indicators age out after almost exactly an hour, almost exactly after 24 hours or 30 days.   I believe part of this is explained based on differences between having the configuration setup as production versus experimental.   I believe another reason for this is because I previously had age out set to first seen + 30 days.   I believe my current problem is that the TAXII node is creating artificial first seen, last seen and age out timestamps based upon when I first setup this instance of DShield within MineMeld.  So while the indicators are in the MineMeld TAXII node, these dates are telling ProofPoint that these indicators are aged out.   I believe what I need is a way to update the last seen and age out timestamps within the miner itself.   I specify the miner because I desire a more complex setup.  If I want several  aggregators making use of the DShield miner as an input then I need the miner to be resolving this issue.   I think..  I'm not entirely sure my analysis is accurate.   Any and all help is appriciated.   Thank you.        ... View more

>>  whitelist some IPs from the indicators extraced from syslog? While not excluding them at the time of extract.  Not all feeds should have these whitelist entries excluded from their final output. ... View more

I believe I see what I had wrong.    Because I hwave whitelist_prefixes set to wl, the processor name itself must also begin with wl.  I had only named the miner begining with wl.   So, I configured a processor with a name begining with wl and bound it to two inputs, the wl miner containing my RFC1918 addresses along with my syslog+7 days miner.  This worked.   Within this same config I have an identical processer with the exception of the name, it does not start with wl.  The output node is identical except it points to this processor.  This one has the RFC1918 addresses.   The point being that with only the name of the processor changing I can see the whitelist working.     I think the following is the relevant config.   I have 300+  lines in the config ATM, so I'm not posting it all.   The end result in this case is that I have PanOS syslog output feeding into MineMeld while whitelisting RFC1918 addresses.  Longer term I could use external dynamic IPv4 lists, such as a list of CloudFlare addresses,  as a feed into a whitelist that I aggregate against the PanOS syslog entries to generate output feeds that I can use in specific dynamic PanOS firewall rules.  After enough hits from an IP, if it isn't on a known list, completely block that IP at every firewall at all locations for a period of time.  This is just one example of many I'm considering.   syslogMiner7Days: inputs: [] output: true prototype: minemeldlocal.syslogMiner7Days aggregatorIPv4GenericStaticWhiteList: inputs: - wlWhiteListIPv4 output: true prototype: stdlib.aggregatorIPv4Generic wlWhiteListIPv4: inputs: [] output: true prototype: stdlib.listIPv4Generic localSyslog7Days: inputs: - syslogMiner7Days - wlWhiteListIPv4 output: true prototype: stdlib.localSyslog wlListIPv4Generic-testingWhiteList20171222b: inputs: - wlWhiteListIPv4 - syslogMiner7Days output: true prototype: stdlib.aggregatorIPv4Generic feedIPv4TestingWhiteList: inputs: - wlListIPv4Generic-testingWhiteList20171222b output: false prototype: minemeldlocal.feedIPv4TestingWhiteList    ... View more

No, although I do have a WildFireEvent miner cloned from localDB as per the article " Using-MineMeld-as-a-Incident-Response-Platform".   Is localDB what I should be using?          ... View more

I have my Palo Alto Networks Firewall intigrated with ProofPoint TAP, TRAP, WIldfire intigrated, MineMeld in use with multiple input/output nodes and aggregators sending data in different directions.      Proofpoint is setup to send parsed copies of questionable yet undetermined e-mails to an IBM X-Force Exchange collection which is email enabled.   These appears as attachments in the collection, which is the lower right corner of their dashboard.   This collection is intigrated, using my API keys,  with VirusTotal, Riskiq.com and many others.   Each of these services I have extended with other security APIs that said service supports out of the box.    At the end of this, e-mail is sent to my IBM X-Force collection and becomes stripped out intel.  IBM has stripped out indicators from these emails and added them to the collection as reports.   These reports send to all of the various intigrations I've enabled.   The results are summerized in the report output.   Or, said simply, I send a copy of a questionable e-mail to IBM, they process it as per how I have things configured, and I receive a TAXII/STIX feed that I can populated back down the the Palo Alto Networks firewall VIA MineMeld.   In my case, IoCs are being shared between many platforms.   Should later on a SHA256  become detected as high confidence malware, I can automate the recall of related content after the fact.  I send APIs requests to my various Antivirus products to request a detailed scan to report back to my software package.  I can send a request to the firewall for all url access for the time period.  I can grab a list of all DNS requests the client has made, looking for VPN over DNS type traffic.   PanOS can consume these lists using MineMeld output nodes.   Anyway, I think that addresses your question and gives some examples on extended the idea.    IBM Threat XChange is free for a basic account.   The product set is enormous, can do anything, but good luck finding a flat price list.  I recommend at least setting up a free account and making use of that.   The free account gives one some very nice feautres for the price.             ... View more

FWIW: regarding this vulnerability, I recently upgraded an HA pair of PA-3050's running 7.0.18 to 7.0.19.   I'm experiencing some issues with SSH connections.  I do not know if this is related or not.   I've also been running into errors causing commit failures.   These are errors I've never seen before though is reminiscent of a problem I recall from the PanOS 4.x days.   I would post the message had I not since cleared the logs.   What I've noticed is this:  LAN->DMZ SSH sessions are being dropped even though the destination IP is whitelisted.   This destination has decryption disabled by the destination IP address.  After many attempts to resolve the issue, SSH began working after I removed all security profiles from the security policy for this destination. LAN->WWW sFTP sessions are being dropped.  The destination is whitelisted and has SSL decryption disabled.  In this case there is no decryption profile assigned to the SSH proxy decryption policy rule.   After many attempts I was able to get this working within the latest version of WinSCP by turning on SSH decryption at the firewall.  Yes, enable decryption.   Yes, this is sFTP and not FTPs.  FileZilla still does not work.   It begins cipher exchange and then the connection drops.   From very limited testing, AIX sFTP does work as does Ubuntu sFTP.   The destination is secure2<.>benefitfocus<.>com on port 22.    The site is apparently running a version of GlobalSCAPE Enhanced File Transfer Server from 2007 (v. 5.1).   This may not be applicable to 8.0.6 and I am not 100% positive that this is a PanOS issue.   ... View more

If I  understand this issue correctly, the problem is that the PanOS 4.x cannot handle the SNI extension that is part of TLS. Amazon Cloudfront added SNI support March 2014.   Amazon CloudFront Adds SNI Custom SSL and HTTP to HTTPS Redirect Features I can't whitelist  the entire Cloudfront range; that would be a very bad thing.  It seems like it could become a big deal. It appears that PanOS 5.0.7 added basic support for SNI.   I suppose it it time for me to upgrade off of the 4.x branch. --- Q. What is the difference between SNI Custom SSL and Dedicated IP Custom SSL of Amazon CloudFront? Dedicated IP Custom SSL allocates dedicated IP addresses to serve your SSL content at each CloudFront edge location. Because there is a one to one mapping between IP addresses and SSL certificates, Dedicated IP Custom SSL works with browsers and other clients that do not support SNI. Due to the current IP address costs, Dedicated IP Custom SSL is $600/month prorated by the hour. SNI Custom SSL relies on the SNI extension of the Transport Layer Security protocol, which allows multiple domains to serve SSL traffic over the same IP address by including the hostname viewers are trying to connect to. As with Dedicated IP Custom SSL, CloudFront delivers content from each Amazon CloudFront edge location and with the same security as the Dedicated IP Custom SSL feature. SNI Custom SSL works with most modern browsers, including Chrome version 6 and later (running on Windows XP and later or OS X 10.5.7 and later), Safari version 3 and later (running on Windows Vista and later or Mac OS X 10.5.6. and later), Firefox 2.0 and later, and Internet Explorer 7 and later (running on Windows Vista and later). Older browsers that do not support SNI cannot establish a connection with CloudFront to load the HTTPS version of your content. SNI Custom SSL is available at no additional cost beyond standard CloudFront data transfer and request fees. Q. What is Server Name Indication? Server Name Indication (SNI) is an extension of the Transport Layer Security (TLS) protocol. This mechanism identifies the domain (server name) of the associated SSL request so the proper certificate can be used in the SSL handshake. This allows a single IP address to be used across multiple servers. SNI requires browser support to add the server name, and while most modern browsers support it, there are a few legacy browsers that do not. For more details see the SNI section of the CloudFront Developer Guide or the SNI Wikipedia article . ... View more

The DNS entry app-assets.plangrid.com cnames to dogqy6r8czetd.cloudfront.net which points to IPs in Cloudfront. The DNS entry app.plangrid.com cnames to gifu-9754.herokussl.com which cnames to elb037100-1517692282.us-east-1.elb.amazonaws.com which points to IPs in Amazon AWS. Both of these use the exact same RapidSSL wildcard certificate. The problem appears somehow related to https://dogqy6r8czetd.cloudfront.net/    With decryption turned on, I can't access this.  With decryption disabled I can. Is this a bug in PanOS 4.1.16?   Or could this be related to SSL Decryption for Some Site Shows as Not Trusted ?? Could someone else running PanOS 4.1.16 with full decryption enabled see if they can connect to https://dogqy6r8czetd.cloudfront.net/  please? ... View more

Thank you. URL logs written:              10537835 Then a little later URL logs written:              10560883 However, there are no URL logs for the last 6 hours in the web interface view. The output from the other command leads me to believe we aren't dropping URL log requests. Current read idx: 23604 Current write idx: 23604 Total URL processed: 89140 Total URL received: 89140 Total URL request loss: 0 Total URL request discard: 0 Current async read idx: 39227 Current async write idx: 39227 Total async URL processed: 39227 Total async URL received: 39227 Total async URL request discard: 0 debug software restart device-server - this did knock off all my Citrix Xenapp/Xendesktop users.   I think this is because I have SSL decryption of this traffic enabled on the PA.  This did not resolve the issue. I'm not certain how to read the logdb-quota output.  If I am reading it correctly, I am full because the threat logs exceed my 19GB qouta, and URL filtering goes to the threat logs.   Quotas:             traffic: 31.00%, 36.932 GB               threat: 16.00%, 19.061 GB               system: 4.00%, 4.765 GB               config: 4.00%, 4.765 GB               alarm: 3.00%, 3.574 GB               trsum: 7.00%, 8.339 GB         hourlytrsum: 3.00%, 3.574 GB           dailytrsum: 1.00%, 1.191 GB         weeklytrsum: 1.00%, 1.191 GB               thsum: 2.00%, 2.383 GB         hourlythsum: 1.00%, 1.191 GB           dailythsum: 1.00%, 1.191 GB         weeklythsum: 1.00%, 1.191 GB             appstat: 6.00%, 7.148 GB               userid: 1.00%, 1.191 GB             hipmatch: 3.00%, 3.574 GB   application-pcaps: 1.00%, 1.191 GB         threat-pcaps: 1.00%, 1.191 GB   debug-filter-pcaps: 1.00%, 1.191 GB         hip-reports: 1.00%, 1.191 GB             dlp-logs: 1.00%, 1.191 GB Disk usage: traffic: Logs: 15G, Index: 16G threat: Logs: 9.3G, Index: 11G system: Logs: 207M, Index: 43M config: Logs: 1.3G, Index: 9.9M alarm: Logs: 72K, Index: 36K trsum: Logs: 4.2G, Index: 4.3G hourlytrsum: Logs: 3.1G, Index: 412M dailytrsum: Logs: 1.1G, Index: 147M weeklytrsum: Logs: 1.1G, Index: 148M thsum: Logs: 2.4G, Index: 79M hourlythsum: Logs: 56M, Index: 42M dailythsum: Logs: 34M, Index: 40M weeklythsum: Logs: 24M, Index: 9.0M appstatdb: Logs: 288M, Index: 125M userid: Logs: 1.2G, Index: 440K hipmatch: Logs: 16K, Index: 16K application-pcaps: 681M threat-pcaps: 4.0K debug-filter-pcaps: 8.0K dlp-logs: 4.0K hip-reports: 1.1M wildfire: 4.0K show log threat - this shows that I have logs going back to 12/11/2013.  I lowered the threat DB from 16% to 7% and then did a commit.  After the commit, an Exec task ran for some time.  After the Exec task completed, I started receiving URL filtering logs again.  I believe reducing the threat DB caused the database purge to occur. I increased the threat DB back to 16% and did another commit.  This seems to be a work around for this issue. ... View more

Perhaps I'm looking at this wrong.   When the URL filtering logs quit logging, what is the appropriate action to take to get URL logging again?   I am assuming that the logs are full and thus I have no logs.   Perhaps that is not the problem at all. ... View more

Thank you. Currently my URL filter has quit logging because (I assume) it is full. >>The quota is checked each time a logdb file is rotated. If the quota threshold is violated then we start deleting logs starting from the oldest until the threshold is no longer exceeded. To see how often the logdb file is rotating, you can review the ms.log file for the following entry "Initing log file with version". Can I manually run the logdb rotation?  If so, how? ... View more

For what it is worth, I have been running Microsoft DNS servers behind the Palo Alto firewall for quite some time.  These are in my DMZ, exposed to the Internet, and allow resolution of a few of our DNS zones.  I'm currently running Microsoft server 2012 on these DNS servers. The Palo Alto firewall rule is nothing special.  It is your typical rule to allow incoming traffic, and allows UDP port 53 as a service.  I have application set to any. I have manually configured bi-directional NAT so that inbound and outbound traffic all originates from and terminates to the same public IP address. One thing that got me when I first setup this up was I forgot to go into the Windows server firewall rules and allow DNS from networks other than the one the server was on. ... View more

I suppose I could purchase two Barracuda Link Balancers, set them up HA, and then use PBF on the Palo Alto to send specific traffic to the inside vIP of the Barracudas.  Such traffic would likely need to be double NATed.  I would need to exclude B2B destinations that have IP filters in place based on our BGP network, or get static IPs on the cable modems and have those added to the IP filters. I suppose I could connect a port from ETHERNET2 to each of the Palo Alto firewalls on a new Ethernet port, then use PBF to redirect traffic to the existing Barracuda.  Such traffic wold be double NATed.   As I've never used the police based forwarding before, I'm a bit concerned about making this change on my production firewall. In this scenario, say I gave the Palo Alto Firewalls a 192.168.0.9/24 address on the private inside of the Barracuda Link Balancer.  I assign it as zone WIFI and interface 1/7.   Then I create PBF rule that says if the source is (PC VLAN RANGE, Zone TRUST) and the destination is (Akamia CDN Netblock)  I forward this to egress from interface 1/7 (right?) with a next hop of 192.168.0.1, my Barracuda inside vIP.   I would create a fail-over monitor and specify an IP address to ping.  Preferably, the IP address would be a remote IP address at Akamia, not something on my LAN.  This way I can fail over if there is an upstream failure. Do I understand correctly?  If not, what mistakes am I making?   ... View more

Attached is a small section of my rules.   These rules have profiles associated with them which include URL filtering, Data filtering, and so on.  Some groups have Google Earth, Google Maps, both, or none. If I stick a rule above all these rules which allows Google Earth and Google maps, then I'm breaking my URL filtering as the allow rule would allow this traffic regardless of what the subsequent rules state. I would need to duplicate every single one of these rules that allow Google Earth and/or Google Maps and set the top duplicate rule to allow the platform without data filtering, correct? ... View more

For what it is worth, our 2050 HA Pair reboot quite often using PanOS 4.1.10.   I'm currently reviewing the forums for issues with 4.1.12 prior to upgrading. ... View more

Thank you. dogbert@PA-2050-Trailer-Rebuild(active)> show system resources | match srvr 2360       20   0  651m 106m 3192 S    0 10.9 535:48.96 mgmtsrvr 2381       20   0  422m 106m 9912 S    0 11.0 610:47.25 devsrvr It sounds like this means PA2050 is underpowered for my needs.  Since I have an HA pair, perhaps the better thing to do would be to completely reboot the main unit so that the passive one takes over.   That way I wouldn't loose the ssl-terminations or user to IP mappings.   My shop is heavy on inbound and outbound SSL termination, and just about every outbound allow rule is based on active directory user mapping. Thank you. ... View more