Tips for Avoiding Alert Fatigue in the Cloud

[rotto]

As a veteran Prisma Cloud Customer Service Engineer (CSE) with Palo Alto Networks, I've noticed some patterns with customers who are successful in their adoption of Prisma Cloud. We're going to talk about one in particular today: how to avoid alert fatigue.

Alert fatigue is a common hurdle for new customers. When so many alerts are being generated that they are a nuisance or ignored, they become near-meaningless or even frustrating. In this situation, it’s easy to feel lost and directionless. Luckily, there are a few things that you can do to avoid alert fatigue from the start.

If you are at the beginning of your journey with Prisma Cloud—or are looking to re-evaluate your current environment—I recommend you start by thinking of your end-goal. Ideally, you want actionable and relevant alerts delivered directly to the people responsible for them. Ask yourself a few questions, like: Who is handling these alerts? What accounts or clouds are they responsible for? What method do they want to use to receive these alerts? What security policies are their top priority? Are certain teams only responsible for certain services? Once you are in this mode of thinking, we are ready to approach one of the key pieces of Prisma Cloud—the account group. 

Account groups are a simple concept. However, using account groups in a thoughtful fashion will have a big impact on two other areas of Prisma—alerting and RBAC. Mismanaging account groups, by either having only one default account group or grouping accounts in ways that aren’t meaningful to RBAC and alerting, can impact the value of the alerts that are generated and complicate overall access. To avoid this, first generate—and answer—questions like I've offered above. If you find yourself in a situation that isn't quite right, don't fret! Account groups can easily be changed later.

Once you have the account groups ironed out, you can start assessing which policies will be tied to which groups. Policies inform alerts. End-goals will vary, from meeting a specific compliance standard, creating a custom one, or focusing on certain services. An additional option would be to create an alert rule for auto-remediation; some Prisma Cloud policies have the option to be automatically remediated if you choose to give extra required permissions. If you set up an alert rule and select the auto-remediation option, Prisma will take care of the alerts for you by handling the violations as soon as they are noticed. 

If you are still seeing an overwhelming amount of alerts, there are a few things to consider. First, look at the policy (or policies) generating the most alerts. Does the policy make sense for your organization? Or just for certain account groups? For example, if you are seeing a wealth of alerts tied to unencrypted volumes and know that this is something that is not currently a priority (or may never be a priority), then you would likely be better off disabling the policy. Disabling the policy will resolve associated alerts and remove them from your total. If the policy turns out to be something you want to keep track of, you can always re-enable it—and the alerts will regenerate. In other situations, it might make sense to take existing policies, modify the RQL to better fit your environment or use-case, and replace the default policies with them. 

Discover more how-to materials, technical documentation, and product updates on LIVEcommunity's Prisma Cloud page.

As always, feel free to drop us a comment below.