Hello Live Community!
Have you ever encountered a scenario where a single, high-bandwidth UDP stream (an "elephant flow") caused a spike in your firewall's packet buffer utilization? When migrating workloads or upgrading hardware, understanding how the data plane processes these massive single-pass streams is critical to maintaining peak performance and network stability.
This article provides an architectural breakdown of how Palo Alto Networks Next-Generation Firewalls (NGFWs) handle single-flow UDP streams, details how to leverage Packet Buffer Protection (PBP), and outlines a step-by-step workflow to safely offload trusted flows.
Technical Prerequisites & Architectural Context
- Single-Core Flow Pinning: Standard Layer 3/Layer 4 session hashing maps identical IP 5-tuples to a single data-plane processing core/flow engine. Consequently, a single multi-gigabit UDP stream cannot be load-balanced across multiple cores.
- UDP Behavior & Content Inspection: Unlike TCP, UDP lacks built-in windowing or flow control. When Security Profiles (Threat Prevention, IPS, Anti-Spyware) are attached to an unclassified (unknown-udp) multi-gigabit flow, packets are held in descriptor queues awaiting Content Inspection / Pattern Matching (the CTD engine)—particularly during abrupt session teardowns and immediate restarts.
- Gen5 Hardware Considerations: Gen5 NGFWs with upgraded ASIC pipelines, higher single-thread clock speeds, and substantially larger packet descriptor pools. However, hardware upgrades alone do not eliminate single-core processing constraints if payload scanning remains enabled on a single multi-gigabit unclassified UDP stream.
- Zone/DoS Protection vs. Packet Buffer Protection (PBP):
- Zone & DoS Protection: Primarily mitigate flood rates (e.g., PPS spikes of new sessions) and will drop ingress packets early rather than helping inspect legitimate, established elephant flows cleanly.
- Packet Buffer Protection (PBP): Actively protects the firewall from device-wide lockups by applying Random Early Drop (RED) or drop/block actions to an offending elephant session once shared buffer utilization (or latency) crosses its threshold .
Option 1 (Recommended): Offload Trusted Elephant Flows via Custom App-Override
When a multi-gigabit UDP stream comes from a known, trusted internal or telemetry source and does not require Layer 7 payload inspection, offloading the flow to hardware FastPath eliminates descriptor queue buildup and achieves line-rate throughput with zero buffer stress.
Step 1: Create a Custom Application Object
Note: Do not use the predefined unknown-udp system application in an App-Override policy; always define a dedicated custom application object.
- Navigate to Objects > Applications and click Add.
- Name the custom application (e.g., custom-high-bw-udp-stream) and assign an appropriate Category, Subcategory, and Technology (e.g., network-protocol).
- Under Advanced > Defaults, select Port and specify the exact protocol and destination port (e.g., udp/<port>).
- Click OK.
Step 2: Configure the App-Override Policy
- Navigate to Policies > App Override and click Add.
- On the General tab, provide a descriptive rule name (e.g., Offload-Telemetry-UDP-Flow).
- On the Source and Destination tabs, restrict the rule strictly to the known source/destination zones and IP addresses of the elephant flow.
- On the Protocol / Application tab, select the custom application created in Step 1 (custom-high-bw-udp-stream).
- Click OK and move the rule above broader App-Override rules if applicable.
Step 3: Commit and Verify FastPath Offload
- Click Commit in the top-right corner of the WebUI.
- Once active, verify via the CLI that the session is identified as your custom application and successfully offloaded to hardware without CTD inspection overhead:
show session all filter application custom-high-bw-udp-stream
show running resource-monitor
Option 2: Tune Packet Buffer Protection (PBP) Thresholds
If you must keep Layer 7 inspection enabled, or need to prevent premature drops during brief micro-bursts while keeping global firewall stability intact, tune Packet Buffer Protection rather than disabling it . Disabling PBP altogether allows packet buffers to reach 100% utilization, risking device-wide unresponsiveness and impacting all legitimate production traffic.
Adjust Zone-Level Packet Buffer Protection
- Navigate to Network > Zones and select the ingress zone receiving the high-bandwidth traffic.
- Under Packet Buffer Protection, enable Packet Buffer Protection (or verify your assigned Zone Protection Profile) .
- Navigate to Device > Setup > Session > Packet Buffer Protection to review or tune the global thresholds and timers :
- Alert (%): The threshold (default 50%) at which the system starts logging buffer utilization alerts .
- Activate (%): The threshold (default 80%) at which the firewall actively applies Random Early Drop (RED) to the packets of the most abusive sessions .
- Block Countdown Threshold (%): The threshold (default 80%) that starts the countdown timer before a blocking action is implemented . Best practice is to configure this 10% lower than your Activate threshold (e.g., 70%) to gracefully trigger host blocking before buffer saturation peaks.
- Block Hold Time (sec): The duration (default 60s) an offending session is allowed to continue under RED mitigation before the firewall blocks the session or quenches the source IP .
- Block Duration (sec): The quarantine period (default 3,600s) during which the offending session remains discarded or the source IP address is blocked .Click OK and Commit the changes.