Cortex XSOAR: The Low Hanging Fruits of Phishing and Spam Detection

[JStephan]

As we all know and face constantly, spam and phishing are dominating email communication in the modern world. Some of the emails are obviously to the human eye and can be easily categorized by at least one human. But what can we do if we have the urge to categorize those emails automatically? How can we determine something is spam over all others?

First, do not try to start with challenging cases first, instead, we should first aim towards getting the easy ones out of the way.

Determine True Spam

Source: Palo Alto Networks XSOAR Marketplace | Marketplace (pan.dev)

Spam might be the easier ones, but as people tend to report a lot of emails, taking the spam out of the equation can reduce the number of cases tremendously. To do that we should take the help of some well-known third party enrichments

• Spamhaus – The Spamhaus Project is an international organisation based in the Principality of Andorra, founded in 1998 by Steve Linford to track email spammers and spam-related activity. The name spamhaus, a pseudo-German expression, was coined by Linford to refer to an internet service provider, or other firm, which spams or knowingly provides service to spammers. (Source: Wikipedia)
• Spamcop – SpamCop is an email spam reporting service, allowing recipients of unsolicited bulk or commercial email to report IP addresses found by SpamCop’s analysis to be senders of the spam to the abuse reporting addresses of those IP addresses. SpamCop uses these reports to compile a list of computers sending spam called the “SpamCop Blocking List” or “SpamCop Blacklist” (SCBL). (Source: Wikipedia)
• Abuse.ch – abuse.ch’s main goal is to identify and track cyber threats, with a strong focus on malware and botnets. We not only publish actionable threat intelligence data on cyber threats but also develop and operate platforms for IT security researchers and experts enabling them sharing relevant threat intel data with the community. (Source: Abuse.ch)

All these enrichments can let us know if indicators of an email are already known to the spam hunter community. If these are known we can easily (and automatically) come to the conclusion that the email in question is real spam and most likely nothing else.

In addition we should also filter the emails which do not follow the best practice of modern email communication. To do that we have at least two tools:

• MxToolBox – This test will list MX records for a domain in priority order. The MX lookup is done directly against the domain’s authoritative name server, so changes to MX Records should show up instantly. You can click Diagnostics , which will connect to the mail server, verify reverse DNS records, perform a simple Open Relay check and measure response time performance. You may also check each MX record (IP Address) against 105 DNS based blacklists . (Commonly called RBLs, DNSBLs) (Source: MxToolBox)
• Spamassain (Postmark) – Have you ever wanted to process the spam score of incoming or outgoing email messages, but didn’t want the hassle of managing SpamAssassin? Now you can use our lightweight JSON API and instantly integrate spam score processing in your app.(Source: Free JSON API to instantly check the spam score of your email messages (postmarkapp.com))

Determine Friendly Emails

Over the duration of your SOC optimization you should find a way to build your list of friendly emails. Friendly emails as an example could be the notification of your vendors. It makes sense to add these sender domains to a list and at least treat them differently. Of course, if all indicators are malicious this is still bad and we should add this check down the line to not give these senders an “access all areas” free pass.

Of course it is up to you what you consider as friendly emails, but all I am saying is that you should keep track and start listing them. As an example:

• Microsoft.com / Google.com – I generally ignore emails from bigger companies where I get regular emails from. These are newsletters or notifications (like github.com). Endusers could be bothered by them anyway as some may consider them as unwanted email, but that does not mean that we should waste resources on analyzing them.
• essent.nl – As one example of a supplier. In a company environment this definetly look different but maybe think along the lines of Dropbox.com / T-Mobile / Nederlandse Spoorwegen
• Internal Emails – Emails from your own domain could be in most cases considered as friendly emails. But keep in mind that an email address could be spoofed or the account could be hacked, so there should be at least some verification on the email sender and the email. On the other hand, depending on your other sources of intelligence maybe a good idea is to look for any hacking indication like impossible traveler, login ip addresses and such

Also keep in mind that these are meant to be the sender and origin email domains, not links within the email. Especially file sharing platforms might host malware or phishing.

Look Deeper

Email headers are a wonderful source of information. The path an email took from one point on the world to your email inbox is maybe the best documented ever. Every single server on the way will add its mark to the paper trail. We should definitely try to dismantle all these information and start looking into these.

XSOAR mapping of fields

• First Server IP address : from autucherry.dz3.reulalo.pinueetrocarloleo.best (92-244-117-254.kievnet.com.ua. [92.244.117.254]) by smtp.gmail.com 
  This indicator will help you further investigate the origin of the email.
• Sender Domain
  Return-Path:<autucherry.dz3@reulalo.pinueetrocarloleo.best>
  From:Milani Romero <autucherry.dz3@reulalo.pinueetrocarloleo.best>
  Always try to get the sender domain and maybe look into the From and the Return-path as well
• First Server Owner – This is mainly an enrichment of the “First Server IP address”, here we use whois to identify the AS and Owner of this IP address. Often we see that several spammers, although the use different domains, use the same origin server. Blocking of these servers after a certain threshold could be a good idea.
• First Server Location – This can also further help. If your business mainly operates in a certain geo location, origins of emails from different parts of the world could give a decent indication.

Revisit the Way You Receive Email

After doing all of this you should take your lessons learned and go back to the way you are doing email communication. Especially on DNS (Domain Name System) level we have developed great baseline protection over the past years (Sources: Wikipedia)

Domain-based Message Authentication, Reporting and Conformance (DMARC) is an email authentication protocol. It is designed to give email domain owners the ability to protect their domain from unauthorized use, commonly known as email spoofing. The purpose and primary outcome of implementing DMARC is to protect a domain from being used in business email compromise attacks, phishing email, email scams and other cyber threat activities.

Once the DMARC DNS entry is published, any receiving email server can authenticate the incoming email based on the instructions published by the domain owner within the DNS entry. If the email passes the authentication, it will be delivered and can be trusted. If the email fails the check, depending on the instructions held within the DMARC record the email could be delivered, quarantined or rejected.

DMARC extends two existing email authentication mechanisms, Sender Policy Framework (SPF) and DomainKeys Identified Mail (DKIM). It allows the administrative owner of a domain to publish a policy in their DNS records to specify how to check the From: field presented to end users; how the receiver should deal with failures – and provides a reporting mechanism for actions performed under those policies.

DMARC is defined in the Internet Engineering Task Force‘s published document RFC 7489, dated March 2015, as “Informational”.^[1]
(Source Wikipedia).

Sender Policy Framework (SPF) is an email authentication method which ensures the sending mail server is authorized to originate mail from the email sender’s domain.^[1][2] This authentication only applies to the email sender listed in the “envelope from” field during the initial SMTP connection. If the email is bounced, a message is sent to this address,^[2] and for downstream transmission it typically appears in the “Return-Path” header. To authenticate the email address which is actually visible to recipients on the “From:” line, other technologies such as DMARC must be used. Forgery of this address is known as email spoofing,^[3] and is often used in phishing and email spam.

The list of authorized sending hosts and IP addresses for a domain is published in the DNS records for that domain. Sender Policy Framework is defined in RFC 7208 dated April 2014 as a “proposed standard”.^[4]