Issue
The URL block page does not work for web sites when the web browser has cache information before applying it.
Details
If the user accesses a website before the URL block page is implemented, the URL block page will not be applied if the users web browsers cache already has the site to be blocked in its cache. For example, apply the URL filter block page for "streaming media" category and access to http://gyao.yahoo.co.jp/korean/ and http://gyao.yahoo.co.jp/ct/music/.
> test url gyao.yahoo.co.jp/korean/
gyao.yahoo.co.jp streaming-media (Base db)
> test url gyao.yahoo.co.jp/ct/music/
gyao.yahoo.co.jp streaming-media (Base db)
Both are categorized as streaming-media as shown above, but block page does work for http://gyao.yahoo.co.jp/ct/music/
Look at the following HTTP request and response header for each site, see the difference on the response header provided by the web server. There is "Cache-Control" attribute and "Pragma: no-cache" on response, the server will not use web a cache object store on the client's browser. But there is no such cache-control for http://gyao.yahoo.co.jp/korean/, so the client browser will use cache rather than accessing to the site, hence the block page will not be supplied by Palo Alto Networks firewall.
=====================================================================
HTTP request and response header for http://gyao.yahoo.co.jp/korean/
=====================================================================
GET /korean/ HTTP/1.1
Host: gyao.yahoo.co.jp
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:26.0) Gecko/20100101 Firefox/26.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: ja,en-us;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
Referer: http://gyao.yahoo.co.jp/korean/
Cookie: B=6rahmo59c29vd...
Connection: keep-alive
HTTP/1.1 200 OK
Date: Tue, 21 Jan 2014 06:03:59 GMT
P3P: policyref="http://privacy.yahoo.co.jp/w3c/p3p.xml", CP="..."
Cache-Control: public
Vary: Accept-Encoding
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
Content-Encoding: gzip
=====================================================================
HTTP request and response header for http://gyao.yahoo.co.jp/ct/music/
=====================================================================
GET /ct/music/ HTTP/1.1
Host: gyao.yahoo.co.jp
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:26.0) Gecko/20100101 Firefox/26.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: B=6rahmo59c29vd...
Connection: keep-alive
HTTP/1.1 200 OK
Date: Tue, 21 Jan 2014 06:04:02 GMT
P3P: policyref="http://privacy.yahoo.co.jp/w3c/p3p.xml", CP="..."
Cache-Control: public
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Last-Modified: Tue, 21 Jan 2014 06:04:02 GMT
Cache-Control: private, no-store, no-cache, must-revalidate
Cache-Control: post-check=0, pre-check=0
Pragma: no-cache
Vary: Accept-Encoding
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
Content-Encoding: gzip
Workaround
Clear the cache on the client browser cache, then block page will work as designed.
See the links below for examples on how to clear the cache on the designated browsers:
owner: kkondo