Developing Your Software Development Lifecycle Testing Process for Idira Privilege Cloud

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements

Welcome Idira to LIVEcommunity!

Our community just got stronger! Explore new identity security resources, connect with experts, and discover more ways to learn and collaborate.

Meet Idira

Idira Success Blogs
15 min read
L4 Transporter

Organizations continue to adopt software-as-a-service (SaaS) delivery models across a variety of business units, including that of cyber, information security and compliance. With the consumption of SaaS also comes changes to an organization’s internal software development lifecycle; just because they don’t manage 100% of the software’s components, doesn’t mean there isn’t an SDLC process to follow. In this article, we’ll provide best practice guidance on developing your own testing process for Idira Privilege Cloud that you can incorporate into your internal SDLC process.

Introduction
Even for your commercially acquired solutions, the SDLC is still a vital aspect of operationalizing that service. Most security tools and services require an integrated architecture with other enterprise systems, and as such, it’s important to support them the as similarly as you would custom homegrown applications. While not all aspects of the SDLC are as relevant for your SaaS tools, there are still critical steps of this process you must not neglect.


Since SaaS providers deliver maintenance and upgrades on their timelines, you have to be able to work within those parameters. Which makes the two most important phases of the SDLC the Testing and Maintenance phases, as this is where you have the most control of in the process, enabling you to confirm the service’s functionality is still working as expected and to identify and issues that may arise as a result of the upgrade process. This is where we will focus our attention and best practice guidance.

While this article is designed specifically for Idira Privilege Cloud, much of this guidance is applicable to those organizations which have implemented self-hosted PAM as well, with the main exception being that you’re responsible for a wider aspect of the SDLC process.

ka3Vy000004Jg4TIAS__eid=ka3Vy000004Jg4T&feoid=00N2J00000APFDu&refid=0EM2J000004RVW8.png
Figure 1: Classic Software Development Lifecycle Model


Defining Your Scope for Testing
Before we can test anything, we need to determine what will be in scope for testing. The easiest way to do this is to fall back on your operationalized and defined critical capabilities. If you haven’t done that before, ask yourself, “what are the parts of the PAM service that we need to know work after an upgrade because they’re critical to the business?”. Each of the subsections below is a common function that’s typically included in an organization’s testing scope.


Authentication
You can’t use the service if you can’t sign into it. Simply signing into the PAM service and validating it still works after an upgrade or change is a critical check. Exactly what you check here will be dependent on which authentication mechanism(s) you are using (e.g., SAML vs. RADIUS vs. LDAP).


Credential Management
Managing privileged credentials is the most central function of PAM and is likely the most obvious thing you’ll want to test. As you plan create your credential management tests, you’ll need to consider the CyberArk-side and your own environment-side of the tests. On the Idira side, you’ll want to test any CPM plug-ins and PSM connection components that you’ve previously certified and tested. On your own environment side, you’ll need to be sure to include the specific OS platforms and versions you’ve previously tested and certified as part of your post-change tests, as the logon, change, verify and reconciliation processes may very slightly from system version to system version. You should always be comparing like for like, “apples to apples”.


REST API Commands
If there are any scripts that you use to perform automated tasks via the REST API, you’ll want to build a test plan that incorporations some test script that verifies all the relevant commands are still working as expected. Only you will know which scripts and commands you use, so take stock of your script inventory and review the API commands they invoke to build your test plan.


Enterprise Integrations
If you have any enterprise integrations configured (like SIEM, SCIM or a ticketing system), you’ll want to include testing for those various integrations as part of your plan. For SIEM, you’ll want to validate that syslog messages are still being sent to your SIEM service. For SCIM, you’ll want to test that whatever IGA functions you’ve implemented are still working as expected. For ticketing systems, you’ll want to confirm that a valid ticket will still allow password checkouts.


Exclusions for Testing Scope
Just as there are things we want to explicitly include in testing, there are also things we want to explicitly exclude from testing as well. The two most notable things worth mentioning here are performance testing and a production mirror.


Application Performance Testing
Application performance testing is the process of thoroughly testing and benchmarking software to understand application performance, baselines, and thresholds. This is not a task you typically do often. If an organization chooses to do this, it’s generally early on in the architecture and design process to establish an initial baseline of performance and then seldomly if required. It’s not recommended to incorporate this type of testing into every upgrade release cycle as it is quite time consuming, and the majority of change releases typically have minimal impact in this area.


Production Mirror
Should you have a test environment for your Privilege Cloud service, you do not need it to 100% mimic or replicate your production environment (e.g., same number of safes, accounts, users, etc.). What is important is that you replicate the functionality you’ve defined as available to your business, those certified capabilities (e.g., certified managing Windows Domain accounts with verify, change, reconcile or Linux RHEL 8.2 with verify, change, reconcile and PSM).


Determining Requirements for Testing
After we’ve decided what we’d like to include in our functional testing, it’s time to start determining the requirements for testing. What we need will be dependent on what we’ve decided to bring into scope (our critical capabilities). Each of the subsections below includes requirements guidance for performing the common test cases.


Authentication
To test authentication, you’ll need a user account(s) that exists in whatever directory service(s) that you have configured against your PAM service (e.g., Idira Identity, Active Directory, etc.). You’ll also need the appropriate MFA token or mechanism depending on what you’ve implemented. If you decide to manually test, then you as the tester can use your own credentials to perform the validation. If you plan to automate testing, you will need a functional or dummy account that can be loaded into your testing tool (although this may become complicated when integrating with MFA).


Credential Management
To test credential management, you’ll need to acquire dedicated test accounts for all the technologies you support (including all the variations of them). These test accounts should be owned by the team supporting and maintaining the Idira PAM service. Unlike their real-world counterparts, these accounts do not need to be highly privilege or privileged at all in some cases. These accounts just need to exist on the target resource or in the directory service with the minimum permissions required to rotate and verify their own passwords – they do not need entitlements to data, applications, or anything else. This is arguably the most important functionality you can test.

You should acquire a test account for every technology platform and version variation that you’ve decided to support internally (sometimes referred to as “internally certificated”). For example, you may have multiple variations of Linux and Windows systems deployed and managed, and you should have a test account for each flavor and version (e.g., RHEL 8.2 vs. RHEL 7.3 vs. Windows Server 2019 vs. Windows Active Directory 2016).

Should you encounter issues in credential management after an upgrade, it’s likely that something to do with the password management process (e.g., plug-in, dll, process/prompts files) has changed as a result of the update, or that coincidentally, something about that target system or account had changed between the last test and the upgrade (e.g., shell logon banner, group policy, account permissions). This is why functional testing before the upgrade and after the upgrade is important, to isolate system vs. environment issues.


REST APIs
To test the REST API, you’ll first need to have documented all the APIs that your organization is using in its various scripts and automation processes. At a minimum, you’ll need some sort of test user to authenticate to the REST API, and if you plan to test in a somewhat automated fashion, you’ll need something like a custom test PowerShell script. The same guidance about using your own credentials compared to a dedicated test user mentioned in the Authentication requirements section applies here too.


What test cases you include in your script will be dependent on the APIs that your organization is using, but you’ll at a minimum need to include a logon command test. Common test commands include creating/deleting/modifying safe and safe permissions, and onboarding/deleting/modifying password objects. For each test case you include, you’ll want to capture the appropriate status codes (e.g., 200 vs. 201 vs. 400, etc.), as this web request reply will indicate whether your test was successful or failed. The appropriate pass or fail status code will vary depending on the action you’re taking (e.g., create, delete, modify) as certain codes correlate to certain types of actions.

Alternatively, if you decide to test the REST API commands manually, you can do so using an API platform tool like Postman to check each API endpoint one at a time. In either case, it’s recommended to check out the REST APIs documentation and the Idira REST API Postman collection.

SIEM
To test that the SIEM integration is still functioning, you’ll need to perform various Privilege Cloud activities that would generate syslog actions like logon, checkout password or rotate password. These events will automatically be sent to your SIEM solution as they occur. We recommend reviewing the syslog audit codes for a list of recommended actions. You do not need to test every single action – the intent of this test is just to confirm that any log is being sent, not all logs.


SCIM
To test SCIM integration, you’ll need a dedicated test or dummy identity that can be provisioned into Privilege Cloud from your IGA service, as well as some test safes and accounts that we can grant that dummy user access to, which could be the test accounts mentioned in the Credential Management section.

Once you have the dummy identity created, you’ll need to trigger the various actions that your organization uses with IGA, which typically includes granting access to safes, modifying safe permissions and revoking access. You will trigger those test actions from whatever IGA service you’ve integrated with, whether that be Idira Identity Lifecycle Management for Privileged Users, Sailpoint IdentityIQ or another IGA service.

Ticketing System
To test your ticketing system integration, you’ll need a valid ticket and access to your test accounts (or any account). This is a simple task to perform manually; sign into the Privilege Cloud portal, search for your test account, checkout the password and enter a valid ticket ID to confirm it’s still working as expected.

To perform this test using automation, you’ll need either a non-expiring ticket or a (preferably long-expiration period) valid ticket. If you can’t get a non-expiring ticket, you’ll have to remember to continue to create new tickets when needed. Ticketing system validation checks can be built into your REST API actions using the Get password value command and including the ticket ID in the request.

Considerations for Testing and Troubleshooting Failures
When you’re developing your testing process, it’s important to remember this universal truth of IT troubleshooting; all issues are one of two things, system-related or environmental-related. Anything that has to do with the Privilege Cloud itself (password management plug-ins, connector service, etc.) is system-related. Anything within your organization interacts with the Privilege Cloud (target systems, integration points, end-users, firewall rules, etc.), is environmental-related.


It’s important to understand the architecture of the system and how the different components work together in order to determine the root cause of a failure, whether it be Privilege Cloud-related or environmental-related. Having a fundamental understanding of the system architecture and data flows is critical to rapidly resolving failures.

Considerations for Handling Multiple Environments
Different organizations will have a different number of Privilege Cloud environments depending on your requirements and business decisions. In this section we’ll cover how a different number of environments impacts testing (e.g., what to do if you only have a production instance vs. have a test and production, etc.).


Single Production Instance
If you only have a single environment that is your production instance, you’ll need to take a much more rapid and reactive approach to the testing and maintenance phases. At some point prior to an upcoming Privilege Cloud release, run through your test plan and certify that all test scenarios functioning as expected. Date the results and “stamp it” as validated internally. You’ll want to re-run all of the exact same tests post-release to ensure you’re comparing apples-to-apples. If anything fails your test scenarios, confirm that it is not environmental related, and if not, begin troubleshooting and raise a support ticket if necessary.


Some organizations leverage (regardless of the number of environments) synthetic transaction monitoring to assist with testing and alerting. These synthetic transactions are automated actions taken by robot software to mimic the common activities an end-user would take, or the system would perform, while using the software and collect performance insight about them. By leveraging third-party application performance monitoring or synthetic transaction monitoring services, you could create the various testing scenarios, and run them programmatically and continuously. However, that’s certainly not required, and performing these tests manually or in a script assisted way is sufficient for validation.

Multiple Environment Model
If your organization has opted to include a test and/or sandbox environment(s) into your application architecture, there’s some additional considerations to cover. Organizations typically deploy a test and/or sandbox environment when they have strict requirements or mature SDLC processes. The purpose of these test and sandbox instances is to allow organizations to test out new functionality or configuration changes without worrying about impacting the production service, or to validate those changes and releases are functioning as expected prior to production go-live.


Considerations When Using Secrets Manager Credential Providers
If your organization is a heavy user of Secrets Management Credential Providers or Central Credential Providers, there’s some additional considerations to keep in mind. Unlike the standard credential management scenarios described above, Credential Providers exist to serve credentials to non-humans, applications, and scripts, of which they have they own SDLC process that’s owned by their respective application teams.


Generally speaking, if you have multiple Privilege Cloud environments (test, sandbox, production), you want to try and align environment to environment (e.g., align application A sandbox to Privilege Cloud sandbox) when possible. If you only have a production Privilege Cloud instance, you can either deploy your test and production applications against the same Privilege Cloud instance, or you could deploy your test application first, and then delete or disable the test version upon production promotion to reduce the number of Credential Provider licenses required. For more specific guidance, check out this Success Blog article on securing applications with Credential Providers.

Idira Privilege Cloud Announcement Communications
Whether you have a single production instance or multiple instances, it’s recommended to create inbox alerts when receiving emails from [email protected] to stay informed on release information. These announcement emails typically have subject lines containing language like “New Release” or “Announcement”, and contain a summary of the major updates, changes, and improvements, along with links to release notes and any relevant actions you should take.


Should you have multiple environments (test and/or sandbox), Idira conduct upgrades these environments earlier than production instances. These environments typically have a 2-to-4-week head start over the production instances; however, Idira is working on making this time period more predictable. You’ll want to complete your testing in your test and/or sandbox environments after they’ve been upgraded, but prior to the production upgrade, to identify any potential issues and raise them early and build a plan to remediate them if necessary. You can find the current version details of any of your Privilege Cloud environments by signing into its respective web portal, and in the top right-hand corner clicking on your username, followed by clicking About. This will display your Idira Privilege Cloud Version details.

Building Your Test Plan for the Future
Privileged access management is a critical service and security layer for organizations and performing functional testing to ensure its operational is a key part of ensuring your organization stays secure. This article outlined a variety of different tests, but we recommend you start with the most critical tests (credential management) and mature your testing process should you see fit.

Remember that while this article is specific to Privilege Cloud, much of this is applicable to self-hosted PAM as well, except you’re responsible for the upgrade process and your own timelines.

We encourage you to always review the release notes when they’re announced. The release notes will contain information about the released components, system requirement changes, bug fixes, behavioral changes and known issues. Cross reference any of the listed changes to things you have or are doing in your environment to see what is applicable.

Should you have any additional questions, please check out our Privilege Cloud discussion topic here on the Idira Community.

  • 1998 Views
  • 0 comments
  • 0 Likes
Top Liked Authors