Validation Skillets can be extremely powerful tools to ensure configuration compliance of all your next-generation firewalls. They can especially be useful in situations where Panorama is not an option. In this post, we'll walk through how to build a simple validation skillet.
Just like building a configuration skillet, we will use the Skillet Builder tools to help us build our validation skillet. If you aren't familiar with the Skillet Builder tools, Get Started Building your first Skillet.
Validation skillets work by comparing the raw configuration objects against a set of rules. If you aren't deeply familiar with the XML structure of the next-generation firewall configuration, I recommend using the web interface debug window. This essentially works by showing you what XPaths and XML fragments are being modified as you make changes in the web interface. To get to the TLDR; version, open a browser tab to your next-generation firewall and log in. Then open another tab and navigate to https://$NGFW_IP/debug, hit the clear debug button to get a fresh start, and click the debug checkbox.
With the debug window now open and cleared, you can find the XPath setting we want to validate. Navigate to the setting you would like to validate in the web interface and make a change. You don't need to commit this change, just change some small value. For example, click a checkbox or change a name.
In this example, I am going to create a validation to ensure telnet and HTTP are disabled on the management interface. I will navigate to settings in the web interface and toggle the values there.
Browse to Device > Interfaces > Management
I checked the Telnet and HTTP options and clicked OK.
Back in the debug tab, once you hit Refresh, you should see a ton of debugging output. Inside, you will see the specific XPath and XML fragments that were modified by changing these values in the web interface.
In this case, the output was in part:
[2020/02/28 15:04:08] user=6556896827576927 <request cmd="get" obj="/config/devices/entry[@name='localhost.localdomain']/deviceconfig/system" cookie="6556896827576927"/> [2020/02/28 15:04:08] user=6556896827576927 Response took 0.038s <response status="success" code="19"><result total-count="1" count="1"> <system admin="admin" dirtyId="17" time="2020/02/28 15:00:44"> <dns-setting> <servers> <primary>22.214.171.124</primary> <secondary>126.96.36.199</secondary> </servers> </dns-setting> <hostname>panos-06</hostname> <type> <static/> </type> <ip-address>192.168.128.4</ip-address> <netmask>255.255.255.0</netmask> <default-gateway>192.168.128.1</default-gateway> <update-schedule> <statistics-service> <application-reports>yes</application-reports> <threat-prevention-reports>yes</threat-prevention-reports> <threat-prevention-pcap>yes</threat-prevention-pcap> <threat-prevention-information>yes</threat-prevention-information> <passive-dns-monitoring>yes</passive-dns-monitoring> <url-reports>yes</url-reports> <health-performance-reports>yes</health-performance-reports> <file-identification-reports>yes</file-identification-reports> </statistics-service> <threats> <recurring> <every-30-mins> <at>2</at> <action>download-and-install</action> </every-30-mins> <threshold>48</threshold> </recurring> </threats> <anti-virus> <recurring> <hourly> <at>4</at> <action>download-and-install</action> </hourly> </recurring> </anti-virus> <wildfire> <recurring> <every-min> <action>download-and-install</action> </every-min> </recurring> </wildfire> <global-protect-datafile> <recurring> <hourly> <at>40</at> <action>download-and-install</action> </hourly> </recurring> </global-protect-datafile> <global-protect-clientless-vpn> <recurring> <hourly> <at>50</at> <action>download-and-install</action> </hourly> </recurring> </global-protect-clientless-vpn> </update-schedule> <snmp-setting> <access-setting> <version> <v3/> </version> </access-setting> </snmp-setting> <ntp-servers> <primary-ntp-server> <ntp-server-address>0.pool.ntp.org</ntp-server-address> </primary-ntp-server> <secondary-ntp-server> <ntp-server-address>1.pool.ntp.org</ntp-server-address> </secondary-ntp-server> </ntp-servers> <login-banner>You have accessed a protected system. Log off immediately if you are not an authorized user.</login-banner> <timezone>UTC</timezone> <service admin="admin" dirtyId="17" time="2020/02/28 15:00:44"> <disable-telnet admin="admin" dirtyId="17" time="2020/02/28 15:00:44">no</disable-telnet> <disable-http admin="admin" dirtyId="17" time="2020/02/28 15:00:44">no</disable-http> </service> </system> </result></response>
If there is a lot of output, you can quickly find what was changed by searching for 'dirtyId'. This will indicate items in the config that are "dirty," which just means uncommitted values.
From this, I can see the modified XPath is:
Now, copy and paste an example validation Skillet to a new file and modify it to suite your needs. I copied one from the Example Skillets Repository.
label: Ensure Telnet and HTTP is disabled per compliance policy
Checks the configuration to ensure telnet and HTTP are disabled
- Example Skillets
So far, I've left variables and snippets both blank. I know from the debug tool which XPath I need for the setting we are interested in. The first step is to parse the configuration for that specific XPath, so I can perform some logic against it. Let's add the first snippet to do just that:
- name: parse config variable and capture xpath as an object
- name: system_object
The key to validation skillets is parsing the config into objects that can then be evaluated using logic. There are a couple of options for how to do this. The most important are 'capture_list' and 'capture_object'. In this case, you want to capture a single object from the XML fragment found at that XPath.
This snippet will place an object in the context with the name 'system_object', which reflects the XML structure found at the given XPath converted to JSON.
The exact structure of the JSON object is what you will test to determine if this config is in compliance or not. To explore the structure in more detail, you can use the Configuration Explorer Tool in the Skillet Builder Tools collection.
This tool is very similar to the API Explorer tool in the PAN-OS next-generation firewall web interface. However, it allows you to experiment with advanced XPath queries.
Once you run this tool, the output will show the exact structure of the configuration for that XPath in both XML and JSON formats.
"login-banner": "You have accessed a protected system.\nLog off immediately if you are not an authorized user.",
Now you can see the XPath you want to check, and you can see the structure of the XML. You are now ready to create our first test. You captured the XPath into an object with the name 'system_object' in the parse step. Now, you can check that object to ensure the value of the disable_telnet element is equal to 'yes'.
- name: test_telnet_disabled
label: Ensure Telnet is disabled on this device
fail_message: Telnet should not be enabled on this device!
test: system_object | element_value('system.service.disable-telnet') == 'yes'
The 'test' attribute uses the Jinja expression language to evaluate the expression as a Boolean. Jinja is useful here as it is a well understood, fairly simple expression language. It also offers a lot of built in operators and filters. Skilletlib also has a set of built in filters to optimize the task of working with a PAN-OS configurations.
This test uses the 'element_value' filter to find the value of the element at the path: system > service > disable_telnet. This syntax is very similar to the normal Jinja variable lookup syntax. However, the PAN-OS configuration causes a few wrinkles that necessitates a slightly more complex syntax. Namely, the use of '@attributes' names and names containing '-'. Because of this, we had to build our own variable lookup syntax. For the most part, you can treat this exactly like the normal Jinja variable look up syntax.
While we are here, let's go ahead and add another test as well. This time, we will ensure the HTTP service is also disabled.
- name: test_http_disabled
label: Ensure HTTP is disabled on this device
fail_message: HTTP should not be enabled on this device!
test: system_object | element_value('system.service.disable-http') == 'yes'
Now, you are ready to test your skillet. In the Skillet Builder Tools collection, there is a Skillet Test Tool that is useful for this task.
After pasting in the skillet and entering credentials for a running firewall, you will see the validation results.
Next, just to verify, change the next-generation firewall configuration back to have these values disabled and run the validation again.
Just for fun, I only disabled 'HTTP' and left 'Telnet' enabled.
After running the Skillet Test Tool again, I can see the Telnet check still failed while the HTTP check now passes. Great!
In summary, Validation Skillets can be used to verify configuration from very simple to very complex all using a simplified Jinja expression language and a few custom filters. Be sure to check out all the examples to get more ideas about how you can build your own validations for any use case!