Blog written by the Cortex Research team: Yinnon Meshi, Lior Rochberger, Eran Tamari, and Daniel Frank and published on July 3, 2024.
When dealing with sophisticated and evasive cyberattacks, one of the more challenging aspects for defenders is detecting and preventing the execution of shellcode. Shellcode is a common yet elusive component that is being utilized by threat actors, due to its stealthy nature.
To address the detection and prevention of shellcode, Palo Alto Networks developed a novel AI-based module, specifically designed to tackle the evolving landscape of shellcode threats. By implementing advanced machine learning techniques, our new AI-based module now offers Palo Alto Networks Cortex customers enhanced shellcode prevention capabilities protecting against new and unseen variations of shellcode.
In this write-up, we will dive into the realm of shellcode and examine how our AI-driven approach is once again raising the bar when it comes to threat detection and prevention.
Shellcode is a headerless arbitrary sequence of bytes, representing assembly instructions that are commonly loaded in memory without being backed by a file on disk. In Windows, shellcode is naturally not loaded by the Windows loader, which requires it to run as position independent code and resolve its own APIs.
Threat actors often tend to weaponize shellcode in various post-exploitation phases, using either known exploitation frameworks and payloads like Metasploit’s Meterpreter, or using completely new and custom shellcode payloads. Post-exploitation shellcode commands are delivered by malicious loaders and injectors that commonly allocate memory and trigger execution.
In the past, shellcode was used only to obtain a remote shell, hence the name “Shell-Code”. Nowadays, shellcode is used to perform a wide range of tasks, from creating a remote shell and communicating with a remote server, to loading additional resources and binaries.
The capability implemented in this module focuses on post-exploitation shellcode detection, which is normally used to execute additional payloads after the attacker obtains code execution privilege on the compromised endpoint.
Due to its elusive nature, shellcode presents different and significant detection challenges. Some of the noteworthy challenges include:
The development of this module required us to use a new and innovative approach. We started from an extensive analysis of numerous malicious files, and then we constructed a dataset of the most difficult-to-detect shellcode buffers we encountered during the analysis.
During the development of our AI-based module, our approach faced several key aspects that needed to be addressed:
To effectively address the challenges of detecting and preventing shellcode attacks, we developed a robust solution that leverages advanced machine learning and deep system visibility. Our approach combines cutting-edge techniques with the unique capabilities of the Cortex platform to identify and neutralize shellcode threats in real-time.
By utilizing the power of AI, we can analyze intricate patterns and behaviors that traditional methods might miss. This section will detail how our innovative module functions, providing a comprehensive look at the technologies and methodologies that drive our enhanced protection capabilities.
To implement an effective solution, we utilized the Cortex platform's kernel-mode system call interception, which provides deep visibility into system operations. This capability allows us to detect low-level malicious activities inspected from the kernel.
Running in the context of the Cortex kernel-mode system call interception capabilities, our proprietary ML module can detect and prevent sophisticated malware that employs advanced evasion techniques, such as direct syscalls and custom shellcode obfuscation. This approach not only improves coverage by identifying patterns and behaviors that traditional rule-based memory scanners often miss or will most likely be unable to detect, but also reduces false positives and preserves precision, while ensuring that legitimate processes are not incorrectly flagged as threats.
Finally, our machine learning models continuously adapt to recognize new and unseen shellcode variations, providing a dynamic and robust defense mechanism.
To address the aforementioned issues we faced, we developed a novel ML framework composed of four distinct algorithms tailored specifically for this module.
This framework combines several approaches:
Key features of this framework include:
By combining these methods together into a single framework, we provide our customers an extended protection against sophisticated malicious shellcode. This integrated approach ensures that our AI-based module offers both precise and robust detection, adding a critical layer in cybersecurity defenses.
The following examples exhibit real-life attacks that were caught in our telemetry by the newly developed AI Shellcode module.
Miners are a good example of an evolving threat that is here to stay, and it often employs sophisticated techniques to increase infection rates and persistence. Notably, the authors of miners can utilize shellcode to execute their payloads in memory, in an attempt to avoid detection by traditional file-based antivirus solutions.
When we ran the new AI Shellcode module on our telemetry, it soon caught an ongoing campaign spreading an XMRig-based miner, masquerading as cracked versions of different security software. The campaign, which started at the end of 2023 and continued to the middle of 2024, spread a loader of an XMRig Miner that utilized shellcode injection in order to execute the miner in memory.
Breaking down the different stages of the attack. In the first stage of the attack:
In the second stage of the attack, updater.exe is executed and creates a file called gcvywecf.tmp, which is the XMRig miner payload itself. The miner then injects into two separate instances of conhost.exe:
Figure 2 below depicts the second stage of the attack.
Figure 2. Second stage execution of the XMRig miner, as shown in Cortex set to detect-only mode, for research purposes.
The shellcode detection of the current use-case was generated automatically using our ML models. A snippet of shellcode extracted from a larger buffer in memory is shown in Figure 3 below.
Figure 3. A snippet of the XMRig shellcode
When analyzing the shellcode, a human malware analyst may notice that the malicious code is searching for the MZ and PE headers, indicative of Windows executable files. However, building a native detection system based solely on these patterns would result in numerous false positives in a production environment.
Our ML model goes beyond these obvious indicators. It incorporates additional, seemingly unrelated conditions that might not make immediate sense to a human analyst.
By analyzing complex patterns and correlations within the data, the model generates highly accurate prevention rules. This advanced approach ensures the effective detection of shellcode with minimal false positives.
Figure 4. End user notification for the prevention of the initial loader
Quasar RAT is an open source .NET malware that is used by a variety of threat actors.
Over the past few years, the malware was reported being distributed using different methods, including via other malware and exploitation of vulnerable and unpatched internet-facing servers and applications.
In May 2024, we investigated a Quasar RAT infection, whose infection vector was the exploitation of vulnerable SQL servers. In this case, the attacker used PowerShell to download different components including binaries, scripts and configuration files from a known Quasar RAT command and control (C2) server. The certificate of this C2 can be seen in Figure 5 below.
Figure 5. Quasar RAT’s certificate used for the command and control server
The payloads that were delivered in this campaign were saved in the compromised environment under the path C:\Users\Public and then executed. All of the payloads were configured to use the same Quasar RAT’s server as their C2. The execution chain of these payloads is depicted in Figure 6.
Figure 6. Exploitation of sqlsrvr.exe, as shown in Cortex set to detect-only mode, for research purposes.
Among the different payloads that we observed, one sample exhibited a suspicious behavior that looked like the execution of a shellcode. The malware, named p.exe, connected to its C2 and downloaded an additional file named 1.bin, which was indeed encrypted shellcode.
After the file was downloaded to the environment, p.exe read it together with another file, url.txt (potentially a configuration file), that was downloaded previously by the attacker from the same C2. The shellcode was then loaded into memory and dropped an additional payload, PetitPotato, a local privilege escalation tool.
The SQL process altered the protection of a memory page using the VirtualProtect API, prompting Cortex to assess a potential shellcode buffer.
Despite the absence of obvious red flags and the fact that the buffer was not resembling any known shellcode frameworks, the Cortex Shellcode AI module effectively detected and prevented the shellcode, as depicted in Figure 7 below.
Figure 7. Prevention of p.exe, as shown by Cortex
Figure 8. Quasar RAT shellcode that could pass as benign
CobaltStrike is a sophisticated threat emulation software that was initially created for red teaming and adversary simulation, but like many other cybersecurity tools, it has fallen into the wrong hands. Cybercriminals and nation state threat actors leverage CobaltStrike for its robust post-exploitation capabilities, such as a dedicated C2 and pivoting and lateral movement within compromised networks.
Another notable feature of CobaltStrike is its ability to generate shellcode payloads, which can be extensively customized. This customization allows attackers to craft unique payloads that evade traditional security defenses. However, the Cortex Shellcode AI module addresses this customization feature by leveraging advanced machine learning algorithms to detect and block even the most subtle and low-detected samples. By analyzing patterns and behaviors rather than relying solely on signature-based detection, our solution can detect and prevent threats posed by customized CobaltStrike payloads.
The sample’s low-detection rates in VirusTotal can be seen in Figure 9.
Figure 9. Low-detection rate CobaltStrike
Set to prevent mode, Cortex prevented the execution of the low-detected CobaltStrike, as shown in Figure 10 below.
Figure 10. End user notification for the prevention of CobaltStrike
The Cortex Shellcode AI module, a Palo Alto Networks Precision AI™ technology, represents a significant advancement in the detection and prevention aspects of cybersecurity by combining novel machine learning algorithms with kernel-mode system call interception to detect and prevent elusive shellcode attacks.
This cutting-edge approach enhances detection coverage and adapts to emerging threats by providing robust protection against even the most advanced and custom shellcode payloads. By integrating multiple algorithms and leveraging a multi-view approach, the Cortex research team has developed a comprehensive solution that addresses the complexities of modern cyberthreats, raising the bar in endpoint security.
