topic Re: Custom App for CRL downloads in General Topics

Custom App for CRL downloads

dennisss — Mon, 25 Feb 2013 21:14:39 GMT

Hi,

I am trying to create a custom app that will match CRL downloads, to allow them without any questions ask. Shouldn't be too hard : on a previous web security gateway, I would match a pattern like the following: "http://([^/:])*crl.*\.crl"

When translated to an app signature, I already know I am looking for two patterns, on the following contexts:

http-req-host-header
http-req-uri-path

Now, my issue is that my pattern are incredibly simple. I am not able to reach the 7 bytes limit with this. One other way would be to match the MIME type in the RSP header I guess... application/pkix-crl or application/x-pkcs7-crl are fine. The issue is with the occasional misconfigured root CAs that still reports text/plain.

So.... Is there anything I can do with this to simply allow CRLs without creating a huge custom category?

Thank you.

Re: Custom App for CRL downloads

mikand — Tue, 26 Feb 2013 06:40:37 GMT

Perhaps creating an URL-filter instead?

Re: Custom App for CRL downloads

dennisss — Tue, 26 Feb 2013 14:59:45 GMT

The URL-filter has severe limitations. Unless I maintain a huge custom category of all the common CRL distribution points on the Internet, I can't see how I would do this. I have two main issues with the URL-filter expressions.

1. matching hosts, the wildcard has to match a complete token. I can't use *crl.*.* -> *crl is invalid because it is not the only character in the token. So, I can't restrict destination URLs based on the hostname.
2. The path is not predictable enough. All I know is that the file name is going to be .crl,, but it might a very long path (many subdirectories) or a short one. ex: /*.crl, /*/*/*/*.crl ... Still, that's not so bad because I could create a dozen of URL filter expressions that would match the file up to 11 directories deep, as an exemple. And that would cover most case... But it still something that could be avoided with a custom appid, or less restrictive URL expressions.

Unless there is something I don't know yet about Pan-OS url-filter expressions...

Re: Custom App for CRL downloads

mikand — Wed, 27 Feb 2013 08:21:06 GMT

Sounds like a case where you should contact the appid team and submit this as a request: http://researchcenter.paloaltonetworks.com/tools/

Re: Custom App for CRL downloads

dennisss — Wed, 06 Mar 2013 22:27:58 GMT

Well, I was able to create a working signature. I just needed to increase my "scope" ; I used the http-req-headers instead of http-req-uri-path. So, I can match on something like ".*\.crl HTTP.*User\-Agent.*, and I looking into the http-rsp-headers for the transaction as well. Simple enough... I feel like it was a worthy exercise after all.

I attached the file for other people to use, and I submitted the request as well.

Thank you.

Re: Custom App for CRL downloads

craymond — Wed, 06 Mar 2013 22:46:13 GMT

Thank you -

That was a very interesting work around, and will be useful.

Re: Custom App for CRL downloads

SRA — Sun, 07 Apr 2013 20:15:46 GMT

You can try to match on the http-rsp-headers for the content type of crl : application/pkix-crl, application/x-pkcs7-crl, etc.

Re: Custom App for CRL downloads

fhu_omi — Mon, 30 Mar 2020 11:10:15 GMT

Hello

I tested this custom APP ID but on PAN 9.1.1 and i think also on 9.0.7 it is not working. As i see, there is a problem in the http-rsp-headers match condition. Without the http-rsp-headers condition it is working. The other way around, only with the http-rsp-headers it does also not working, therefore it seems a problem in the http-rsp-headers condition. I tried to simplify this condition .*((pkix-crl)|(x-pkcs7-crl)|(plain)).* but it does not match. I also tried .*((pkix\-crl)|(x-pkcs7\-crl)|(plain)).* and only pkix\-crl , because my test response is a pkix one, but nothing, the requests alwys recognized as web-browsing. Any ideas?

Re: Custom App for CRL downloads

AlexNC — Thu, 30 Jul 2020 14:15:52 GMT

Hi Everyone

I'm new to the topic of custom App-IDs, i tried this custom App-ID. but when accessing these CRL urls then it never matches this rule. It is either incomplete or web-browsing. I am running 9.1.3 currently.

Has anyone this custom App-ID still running and working?

Thanks for any help and best regards

Alex

Re: Custom App for CRL downloads

AlexNC — Thu, 06 Aug 2020 16:06:36 GMT

I've been testing the custom application and trying to figure out what is causing it to not work anymore.

My knowledge of custom App-ID's is very limited so if there is a mistake within the application please apologize and let me know.

For now I have adjusted the custom App-ID and got it to work (i was testing it out by blocking anything matching that App-ID) and checking if all the CRL URL's are being blocked now.

So far I got the following results:

http://mscrl.microsoft.com/pki/mscorp/crl/Microsoft%20IT%20TLS%20CA%205.crl > working OK > application/octet-stream

http://crl.swisssign.net/EEFD46CAF7275E91BC5AB6E787CD0AFA550A2642 > not working yet
http://crl.swisssign.net/E7F1E7FD2E53AD11E5811A57A4738F127D98C8AE > not working yet

Now I only need to know how to upload a file and i'll gladly share the app

Re: Custom App for CRL downloads

Sec101 — Mon, 21 Sep 2020 17:30:33 GMT

I've removed the rsp section, and only used the GET http-req-headers. The match specified in the above article does not work. It's not the rsp section that is the problem, it is the initial http-req-headers pattern match. What did you do to get the first match working?

In the packet captures taken on the firewall it appears there is a /r/n at the end of the request. Does wireshark add these in, or is this something literally being read by the firewall?

Everything else has been removed except the 1st http-request header, method get:

.*\.crl HTTP.*\r\n will match

.*\.crl HTTP.* does not match

.*\.crl\sHTTP.* does not match

Re: Custom App for CRL downloads

AlexNC — Mon, 21 Sep 2020 17:37:41 GMT

<application version="9.1.0"> <entry name="APP_CRL_VAL"> <signature> <entry name="GET CRL File"> <and-condition> <entry name="And Condition 1"> <or-condition> <entry name="Or Condition 1"> <operator> <pattern-match> <qualifier> <entry name="http-method"> <value>GET</value> </entry> </qualifier> <pattern>.*\.crl HTTP.*User\-Agent.*</pattern> <context>http-req-headers</context> </pattern-match> </operator> </entry> </or-condition> </entry> <entry name="And Condition 2"> <or-condition> <entry name="Or Condition 1"> <operator> <pattern-match> <pattern>((application/pkix-crl)|(application/x-pkcs7-crl)|(text/plain)|(application/octet-stream)).*</pattern> <context>http-rsp-headers</context> </pattern-match> </operator> </entry> </or-condition> </entry> </and-condition> <scope>session</scope> <order-free>no</order-free> </entry> </signature> <subcategory>infrastructure</subcategory> <category>networking</category> <technology>client-server</technology> <description>matches CRL download</description> <risk>2</risk> <evasive-behavior>no</evasive-behavior> <consume-big-bandwidth>no</consume-big-bandwidth> <used-by-malware>no</used-by-malware> <able-to-transfer-file>no</able-to-transfer-file> <has-known-vulnerability>yes</has-known-vulnerability> <tunnel-other-application>no</tunnel-other-application> <tunnel-applications>no</tunnel-applications> <prone-to-misuse>no</prone-to-misuse> <pervasive-use>no</pervasive-use> <file-type-ident>no</file-type-ident> <virus-ident>no</virus-ident> <data-ident>no</data-ident> <default> <port> <member>tcp/80</member> </port> </default> </entry> </application>

This is the app that I got to work and added a 3rd way that I found within the wireshark traces.

But again, this is my first attempt on getting anything working with a custom PAN app-id and I used the base application provided within this Post. Without that I wouldn't have even known where to begin.

Feel free to try it out and maybe/hopefully you find a flaw or something. I wouldn't be surprised if there is something odd in there.

Cheers

Re: Custom App for CRL downloads

Sec101 — Mon, 21 Sep 2020 19:49:29 GMT

I must have been doing something wrong, as I'm getting matches off of this now. Thank you for the quick reply. Appears to be working!

Still not sure why the below will create a match. I udnerstand the http.* But the User\-Agent directly after it doesn't make sense. Unless the firewall is taking "any" of these patterns and making a match against what it sees?

HTTP.*User\-Agent.*

A request I see in a browsers appears to look like:

GET /pkiops/crl/MicCodSigPCA2011_2011-07-08.crl HTTP/1.1

there is only a User-Agent listed as a separate request header:
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:80.0) Gecko/20100101 Firefox/80.0
But it must check any of these fields?

Re: Custom App for CRL downloads

AlexNC — Tue, 22 Sep 2020 06:33:05 GMT

Hi There

To be honest the User Agent part was already in there.
I just took the app, tried it..... tried to so what wireshark says/shows and searched some documents on how the string should look.

if I recall correctly there is only a difference was pattern matching and changing it to "session" instead of "protocol-data-unit" after that it worked and I added the case for "application/octet-stream" since I found some CRL Pages sending it like that.

I don't know or do much with custom App-ID so maybe someone else can weigh in on it.

Regards

Alex

Re: Custom App for CRL downloads

Sec101 — Tue, 22 Sep 2020 15:49:58 GMT

I'm not sure on this, but it looks like octet-stream, might be a fallback for an unknown. BTW-Changing to session did the trick. Makes sense as to why "transaction" wouldn't match this no matter what I was adding. It almost looks as though the firewall wasn't reading that HTTP response. Even though the below post seems to say otherwise:

https://live.paloaltonetworks.com/t5/general-topics/transaction-and-session-of-signature-scope/td-p/45437

Re: Custom App for CRL downloads

AlexNC — Thu, 24 Sep 2020 13:44:42 GMT

Hi There

So are you running the app successfully now in your environment?

I have it deployed on a larger scale now and so far if i add a URL filterping profile with alert for all alloed categories I do see only *.crl files in the URL log hitting. But in traffic Log I have a lot more hits (to the same public IPs) then what I have in the URL log.

Regards

Alex

Re: Custom App for CRL downloads

Sec101 — Wed, 18 Nov 2020 15:03:09 GMT

Yes, appears to be working ok for us.

Re: Custom App for CRL downloads

hebejena — Mon, 01 Nov 2021 08:19:33 GMT

I wonder if even the latest version can do that?

Re: Custom App for CRL downloads

AlexNC — Mon, 01 Nov 2021 08:46:13 GMT

Hi Hebejena

How do you mean "if the latest version can do that" do you mean if there is a app-id or a built in way from Palo Alto to cover this use-case with their latest PAN-OS releases?

Regards

Alex

Re: Custom App for CRL downloads

Sec101 — Mon, 01 Nov 2021 19:28:45 GMT

That would be ideal really. Something that would cover all .crl cases.