In this tutorial, we will cover the following objectives:
Prior to creating an Azure DevOps project, you should have an Application registered in Azure Portal to get the Application Client ID and the password, along with a storage account associated with a resource group. A storage account is used to store the Terraform state.
Figure 01: Create the project in AZ DevOps org_PaloAltoNetworks
We need to create a repository where our Terraform IaC templates are hosted. For this tutorial purposes, we will create a vnet with Terraform that would have yor trace tags inserted.
In the “Repos” menu select “Import a repository” option
Figure 02: Create new repo by importing from GitHub_PaloAltoNetworks
Import the GitHub repository with all necessary files from yor demo GitHub repo.
Figure 03: Yordeomo repo with TF templates and pipeline files_PaloAltoNetworks
If you create a new Azure DevOps organization using the Free tier, you won’t be able to execute parallel jobs to run pipelines on the Microsoft-hosted agent pool and will be getting an error: “No hosted parallelism has been purchased or granted”
Add an agent pool in the Project settings
Figure 04: Add new Agent pool_PaloAltoNetworks
Self-hosted with all permissions granted
Figure 05: Configure Agent pool details_PaloAltoNetworks
Click on the pool to add a self-hosted agent
Figure 06: Select New Agent_PaloAltoNetworks
Install the agent on your local machine following the steps in the “New agent” wizard
Figure 07: Follow Agent installation instructions_PaloAltoNetworks
At the configuration step you’ll need an Org Personal Access Token, to create one click on Org Settings
Figure 08: Create a new personal token_PaloAltoNetworks
Once the configuration is completed and the agent is launched, you should have an agent in status online, listening for the jobs
Figure 09: Self-hosted agent is running and listening for the jobs_PaloAltoNetworks
Create a variable group in the Pipelines Library called state-data with entries for the following:
BACKEND_RESOURCE_GROUP_NAME - Resource group name
BACKEND_AZURE_STORAGE_ACCOUNT_NAME - Storage account name
BACKEND_AZURE_STORAGE_ACCOUNT_CONTAINER_NAME - tfstate
BACKEND_AZURE_STORAGE_ACCOUNT_KEY_NAME - terraform.tfstate
BACKEND_TENANT_ID - Tenant ID
BACKEND_CLIENT_ID - Application Client ID from the service principal
BACKEND_CLIENT_SECRET - Secret from the service principal
BACKEND_SUBSCRIPTION_ID - Subscription ID
Set the:
BACKEND_CLIENT_SECRETto a secret variable by clicking on the lock icon.
You can use Azure cli commands to retrieve required inputs
az login --use-device-code
az group list --query [].name
az storage account list --query [].name
az account showSave the variable group.
In Project Settings create a service connection for Azure Resource Manager type
Figure 10: Select Azure Ressource Manager for new service connection_PaloAltoNetworks
Select “App registration or managed identity (manual)” for the Identity type and “Secret” for the Credential. Fill in all the necessary fields, save it as “yor” services connection, this exact name is used in the pipeline files
First, we need to install the Terraform extension from the Visual Studio Marketplace.
TF extension at VS Marketplace
Figure 12: Install Terraform extension_PaloAltoNetworks
In the Pipelines section, create the pipelines from existing repository files pr-plan.yml and merge-apply.yml respectively.
Figure 13: Select existing pipeline files from the repo_PaloAltoNetworks
Both pipelines use objects created previously - the variable group “state-data” and “yor” service connection, as well as they rely on using a self-hosted agent “yor” created above.
Save both pipelines without running.
Rename the pipelines in the “Pipelines” menu to meaningful identifiers instead of the project name set as id by default.
Figure 14: Rename the pipelines_PaloAltoNetworks
Add a build validation policy to the main branch in the repos menu, select the pr-plan pipeline with all defaults.
Figure 15: Add a branch policy_PaloAltoNetworks
Figure 16: Select the ’pr-plan’ pipeline and leave all the rest as default_PaloAltoNetworks
We place the yor installation script as well as yor cli command to tag the TF template in the pr-plan.yml pipeline file.
Figure 17: Add your trace tagging to the pipeline file_PaloAltoNetworks
It is important to place the yor section before the tf plan file creation, to make sure the TF templates are tagged and will be deployed at TF apply stage along with resources being created with the TF templates.
Details on yor trace integration with different CI platforms can be found here.
Click on the state-data variable group. Click on Pipeline permissions and add the pr-plan and merge-apply pipelines to the group.
We will create a new branch, modify the code in it, commit, and create the pull request, this will trigger the pr-plan pipeline to execute.
Create new branch “vnet-deploy”.
Figure 18: Create New Branch_PaloAltoNetworks
In the files, find the terraform.tfvars and set the correct value for the resource_group_name variable. Then commit to the “vnet-deploy” branch and create the pull request.
Figure 19: Update Resource Group Name to the actual value_PaloAltoNetworks
Figure 20: Create pull request_PaloAltoNetworks
Upon pr-plan pipeline execution, we will be able to complete the pull request and merge with the main branch, this in turn will trigger the merge-apply pipeline to execute creating the vnet in Azure cloud with yor trace tags.
Figure 21: Complete the Pull Request_PaloAltoNetworks
Merge-apply is being executed on merge completion.
Figure 22: Merge-apply pipeline is triggered_PaloAltoNetworks
The TF templates deployed the vnet with yor trace tags.
Figure 23: vnet resource is created in Azure tagged by yor trace_PaloAltoNetworks
Checking the pr-plan pipeline jobs detail, we can see that the yor tags were applied to vnet in the main.tf file included in the tf plan file and then deployed.
Figure 24: The yor trace tags are added to the main.tf file_PaloAltoNetworks
