What is App-IDs, PCAPs and Custom Signatures?
Question
What is App-IDs, PCAPs and Custom Signatures?
Environment
- PAN-OS
Answer
What is App-ID?
Application Identification or App-ID is a main component of Palo Alto Networks devices. It is a patented mechanism presented only on a Palo Alto Networks device and is responsible for identifying applications traversing the firewalls independently of its port, protocol and encryption (SSL or SSH). This identification of applications ensures the success of proper Layer 7 inspections at the packet load level, compared with Palo Alto Networks Application Signatures (today over 2,000 individual App-IDs), Application Protocol Decoders, and heuristics. These elements are responsible for the visibility of this Layer 7 (L7) traffic traversing the Palo Alto Networks firewalls.
The engine behind the App-ID component is driven by a series of pre-determined contexts. These contexts use decoders to help identify applications that have been tunneled within the main application, (for example, Google Talk within Gmail). The applications are categorized and classified by the PAN-OS App-ID engine, allowing proper identification and usage of Application Groups at the security policy level.
During this classification process, Palo Alto Networks defines main applications (Parent App) and some directly dependent (or Child App), which are part of these main applications. For instance, by classifying an App, such as “uploading”, as the Parent App in a newly created App-ID that will use file transfer from the web (browser-based file-sharing). This allows the Child App to be properly identified as part of the Parent “uploading” App, and provides visibility to the appropriate application under the correct categorization.
Even though we classify, categorize, and create several known applications within PAN-OS there are still several applications that are not on the Palo Alto Networks devices database. These applications are called “unknown,” meaning unknown to PAN-OS at that time, but not known to PAN-OS. In these cases, custom App-ID signatures may be created to properly identify and classify them.
How Does App-ID Work?
While traffic is traversing the Palo Alto Networks firewalls, the App-ID engine is always providing constant visibility of the logs (Monitor tab) in PAN-OS, but the sequence before that visibility looks like the following:
The traffic needs to match a security policy and allow signatures. These signatures are applied to the traffic to identify the application/applications based on the applications unique characteristics. If the application is using its standard service ports then "application default" should be used in the Services field. If non-standard ports are used, then those TCP or UDP ports will need to be specified in the Services column of the traffic rule.
If the App-ID engine determines that the traffic is being encrypted (SSL or SSH), a decryption policy needs to be in place that to allow the App-ID engine to inspect the traffic.
PAN-OS is a context-based engine. Decoders, for some known protocols, are also applied and will be responsible to identify other “embedded” applications that maybe tunneled within the protocol (for example, Gmail Google Chat used across the HTTP). Some applications may still try to evade and may not be identified through the signatures and decoders. A heuristics or behavioral analysis may be used to identify the application. If after all these steps the application is not properly identified, it will be classified as “unknown” for further analysis and proper identification from the security operations team. If it is still an unknown application, it can be blocked or not be part of an approved applications list placed in the security policy.
How Does PAN-OS Handle Unknown Applications?
When working with any App-ID adoption process, whether through a Migration Tool or manually by analyzing logs, the first step on adopting App-IDs is to separate unknown to known traffic. The known traffic are the applications already identified on Palo Alto Networks firewall logs. The unknown are subject for analysis and must be properly identified. It must have an App override rule created that will be known as “fast path” if it only contains the service ports and will only use Layer 3 and 4 inspection not going to the Layer 7.
These rules can be used to provide visibility during the investigation process of the unknown traffic. Once the proper packet information is inserted and further analysis is carried on to the TCP Stream, a full Layer 7 App-ID signature may be created and will provide visibility and Layer 7 inspection with no need for an App Override rule. For traffic that could not be identified, further analysis is required. Palo Alto Networks logs may provide valid information during this process.
Knowing the reason why an application was marked as unknown-traffic is key and in PAN-OS there are two main types of classification for unknowns:
- Incomplete data, which happens after a handshake was executed but no data came through before the timeout.
- Insufficient data, which happens when after a handshake is completed, some data is sent through but not enough packets were sent to identify the application.
These cases are usually network related or some unconventional applications that communicates in a singular manner. At this point we know enough about the unknown application, but we need a packet capture (PCAP) to properly identify a pattern within a TCP Stream until this session is closed.
With the PCAP on hand, and after proper analysis, use the application within the network to replicate the traffic. Create a PCAP from the firewall to have enough detail and then establish a proper pattern that will be used by creating a Custom App-ID signature or it might be sent to Palo Alto Networks support and it will be created for you.
Note: More than a single packet stream will be needed.
A custom App-ID needs to be created with the same criteria, all other applications are inserted into the PAN-OS App-ID repository. It also needs proper characteristics, classification, category and sub-category, as well as risk level and service port and timeouts.
How to Create a PCAP
Perform a PCAP in order to help identify the unknown traffic.
Please see the following document and video to learn more about creating a Packet Capture:
The video mentioned above demonstrates how to:
- Configure and run a basic PCAP from the PAN-OS UI
- Download the produced PCAP files
- Open the PCAP files for analysis
Create a Custom App-ID
After you have analyzed your TCP stream and a pattern is found, that is constant and not related to the infrastructure around the payload (MAC addresses, hardware manufacture data, NIC information), we can now use this chunk of data into our new custom App-ID signature.
Note: Use the hexadecimal format in your REGEX .
See Also