Terraform provider inconsistencies and issues with IAM role tags

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Please sign in to see details of an important advisory in our Customer Advisories area.

Terraform provider inconsistencies and issues with IAM role tags

L2 Linker

Hi,

 

I am currently on provider version 1.0.4 (prior to this, I was trialling out CloudNGFW a month ago, on locally built version 1.0.0 of the provider). I do not think it's a provider bug as such, but the issues I am facing pertain to provider block configuration.

Before I list the issues, I would like to provide some info on the pre-requisite steps I have completed prior to deploying NGFW via terraform.

 

1. I have subscribed to CloudNGFW via AWS Marketplace.

2. I have added my sandbox account to CloudNGFW tenant via Web UI and I have run the Cloudformation template. I am using Customer managed endpoints.

3. I have created the Cloudwatch Log groups and log streams matching the names in the CT above.

4. I have switched on programmatic access and followed the guide to create IAM role and policy. I have added all 3 tags on my IAM role as shown below

SZanpure_0-1656353140535.png

With this in place, here are my findings:

 

Scenario 1: Running locally built provider version 1.0.0, I could bootstrap the cloudngfwaws provider as such

 

provider "cloudngfwaws" {
  json_config_file = "./cloudngfwaws_config.json"
}

cloudngfwaws_config.json below:
{
  "host": "api.us-east-1.aws.cloudngfw.paloaltonetworks.com",
  "region": "us-east-1",
  "arn": "arn:aws:iam::account-id:role/palo-ngfw-admin-role",
  "logging": ["login", "get", "post", "put", "delete", "path", "send", "receive"]
}

 

This worked flawlessly, and I did not have to specify any access_key or secret_key. The provider would lookup the AWS credentials automatically (not sure how. Possibly by looking at AWS_PROFILE environment variable or the likes).

 

Scenario 2: Fast forward a month, and today I started working on this project again, and upgraded to provider version 1.0.4 from Terraform registry, instead of building it locally. Below are the issues I noticed:

 

Issue 1: If access_key and secret_key are not supplied in provider_config (json file), I get the following error upon running terraform plan:

 

Error: error configuring Terraform AWS Provider: no valid credential sources for Terraform AWS Provider found. Deprecated

 

This issue was NOT for AWS provider, but for the cloudngfwaws provider. So I decided to add access_key and secret_key into the provider config block, which led me to issue 2

 

Issue 2: After supplying the info above, I ran terraform plan, and got the following error:

 

Account is not successfully onboarded by FMS. Programmatic Access for CloudNGFWGlobalRulestackAdmin role is not supported

 

The clue here is "CloudNGFWGlobalRulestackAdmin role is not supported". My terraform code is rather simple. It creates a standard NGFW, with a local rulestack, that has a local security rule defined underneath. I do not want or have a global rulestack. However, the IAM role created in the pre-requisite steps has the "CloudNGFWGlobalRulestackAdmin" tag associated with it. If that tag is defined, it seems to force a dependency on the NGFW being configured in AWS FMS with a matching policy. This is not mentioned anywhere in the docs as far as I can see, though I might have missed it. 

Another thing to note here, is that I attempted to delete the tag from IAM policy in terraform and tried to run terraform plan, but the NGFW provider was not letting me apply that change (terraform plan failed preventing me from proceeding). So I had to manually login to AWS console and delete that tag from IAM policy. This leads me to the next issue.

 

Issue 3: After deleting the tag via AWS console, and deleting it in terraform code, I ran terraform plan again. This time I got a new error 

 

Error: Error(-1): Unknown path: /v1/mgmt/tokens/cloudglobalrulestackadmin

 

This threw me off because I am not creating global rules anywhere and I had already fixed the IAM role above. So I read through the provider docs, and it turns out that if I supply the provider settings "arn" in config block, it tries to generate/refresh all tokens (localfirewalladmin, localrulestackadmin and globalrulestackadmin). So I removed "arn" and instead supplied "lfa_arn" and "lra_arn" in my config block. This led me to my next issue.

 

Issue 4: Now, after running terraform plan, I got the following error. Please note that my config was supplied in json file.

 

Error: No ARNs were specified

 

This is definitely incorrect, since the ARN's are specified.

 

{
    "host": "api.us-east-1.aws.cloudngfw.paloaltonetworks.com",
    "region": "us-east-1",
    "lfa_arn": "arn:aws:iam::account-id:role/palo-ngfw-admin-role",
    "lra_arn": "arn:aws:iam::account-id:role/palo-ngfw-admin-role",
    "logging": ["login", "get", "post", "put", "delete", "path", "send", "receive"]
}

 

So I read the provider docs again, and the config block resolution seems to look for properties in the provider block first, followed by ENV variables, and then in the json file. So I moved all the above config from json file to provider block directly, and at this point, terraform plan finally gave me an output. I have yet to run terraform apply.

 

The point of all this is to note that changes in backend functionality are breaking the provider, and the documentation does not match the backend. These nuances of access_key and secret_key being mandatory, when to use arn vs lfa/lra arn and IAM role tags effectively being used to suss out centralised vs distributed deployment (and AWS FMS therefore being needed for distributed deployment) need to be documented better. 

 

Please let me know if I have missed some relevant documentation, and I will be happy to read it 🙂

 

Thanks,

Shreyas

8 REPLIES 8

L2 Linker

Ok another update after running terraform apply. The customer managed VPC endpoints are not accepted immediately by the NGFW/Palo trusted account. In my case, after running terraform apply, it took around 5-6 minutes, for the status to change from PENDING to ACCEPTED in NGFW Web UI. This meant that my terraform apply failed when creating routes for the VPC endpoints

 

SZanpure_0-1656358383381.png

 

 

│ Error: error updating Route in Route Table (<route table name>) with destination (<cidr>): RouteNotSupported: Route table contains unsupported route target: <vpc endpoint name>. VPC Endpoints of this type cannot be used as route targets.
│       status code: 400, request id: 6e873cbd-291f-4131-8107-ebf1cb3e3997
│ 
│   with module.firewall-vpce.aws_route.inspection-rtb-tgw-subnet-1,
│   on modules/firewall-vpce/palo_ngfw_routes.tf line 27, in resource "aws_route" "inspection-rtb-tgw-subnet-1":
│   27: resource "aws_route" "inspection-rtb-tgw-subnet-1" {

 

L2 Linker

Hi,

 

@gfreeman Tagging you. Hope that's ok.

 

I completely un-subscribed and re-subscribed via AWS Marketplace and started with a clean slate. On the very first run of my provider, I always get this error below

 

SZanpure_0-1656672584061.png

I believe I am getting this error because the creation of cloudngfwaws resource via terraform finishes way before the actual NGFW has finished creation in Palo Web UI. This is causing the VPC Endpoint Service to be in a "non-ready" state when terraform queries it. Can this please be looked at?

 

FYI, my terraform code for creating Palo NGFW is below

 

resource "cloudngfwaws_ngfw" "test-ngfw-terraform" {
  name        = "test-ngfw-terraform"
  vpc_id      = var.inspection-vpc-id
  account_id  = var.account-id
  description = "Test NGFW created via terraform provider"

  endpoint_mode = "CustomerManaged"
  subnet_mapping {
    availability_zone = var.inspection-vpc-private-subnet-1-az-name
  }

  subnet_mapping {
    availability_zone = var.inspection-vpc-private-subnet-2-az-name
  }

  rulestack = cloudngfwaws_commit_rulestack.commit-test-rulestack-terraform.rulestack
}

resource "cloudngfwaws_ngfw_log_profile" "test-ngfw-terraform-log-profile" {
  ngfw       = cloudngfwaws_ngfw.test-ngfw-terraform.name
  account_id = cloudngfwaws_ngfw.test-ngfw-terraform.account_id

  log_destination {
    destination_type = "CloudWatchLogs"
    destination      = aws_cloudwatch_log_group.test-ngfw-terraform-loggroup.name
    log_type         = "THREAT"
  }
  log_destination {
    destination_type = "CloudWatchLogs"
    destination      = aws_cloudwatch_log_group.test-ngfw-terraform-loggroup.name
    log_type         = "TRAFFIC"
  }
  log_destination {
    destination_type = "CloudWatchLogs"
    destination      = aws_cloudwatch_log_group.test-ngfw-terraform-loggroup.name
    log_type         = "DECRYPTION"
  }
}

resource "cloudngfwaws_commit_rulestack" "commit-test-rulestack-terraform" {
  rulestack = cloudngfwaws_rulestack.test-rulestack-terraform-module.name
}

resource "cloudngfwaws_rulestack" "test-rulestack-terraform-module" {
  name        = "test-rulestack-terraform-module"
  scope       = "Local"
  account_id  = var.account-id
  description = "Test rulestack created via terraform provider"
  profile_config {
    anti_spyware  = "BestPractice"
    anti_virus    = "BestPractice"
    file_blocking = "BestPractice"
    vulnerability = "BestPractice"
  }
}

resource "cloudngfwaws_security_rule" "test-security-rule-terraform" {
  rulestack   = cloudngfwaws_rulestack.test-rulestack-terraform-module.name
  rule_list   = "LocalRule"
  priority    = 1
  name        = "test-security-rule-terraform"
  description = "Test security rule created via terraform provider"
  source {
    cidrs = ["any"]
  }
  destination {
    cidrs = ["any"]
  }
  applications = ["any"]
  category {}
  action        = "Allow"
  logging       = true
  audit_comment = "initial config"
}

resource "aws_cloudwatch_log_group" "test-ngfw-terraform-loggroup" {
  name              = "PaloAltoCloudNGFW"
  retention_in_days = 90

  tags = {
    Name = "PaloAltoCloudNGFW"
  }
}

resource "aws_cloudwatch_log_stream" "test-ngfw-terraform-logstream" {
  name           = "PaloAltoCloudNGFW"
  log_group_name = aws_cloudwatch_log_group.test-ngfw-terraform-loggroup.name
}

 

The code which errors is below

 

data "cloudngfwaws_ngfw" "palo-ngfw" {
  name = var.ngfw-name
}

# Errors for this data resource
data "aws_vpc_endpoint_service" "palo-ngfw-vpce-service" {
  service_name = data.cloudngfwaws_ngfw.palo-ngfw.endpoint_service_name
}

 

One last thing to note, is that if I wait for the NGFW to finish creation, and re-run my terraform code, I get the error below. After this, no matter how many times I re-run the build, I keep getting the same result

SZanpure_1-1656673502853.png

 

L2 Linker

Hello @SZanpure 

Greetings from Palo Alto Networks!

Issue 1 We are working on removing the necessity to specify the access key / secret key in the provider block. Not sure yet when it will come out, but it's on our list.
Issue 2 - The API itself has changed how it behaves. We are working on an update for the provider.

Issue 3 - This happens because you removed the global rulestack tag, so it's going to error.
Issue 4 - Looks like you using underscores in the JSON config, if you want to do JSON config, you need to use hyphens.

Regards,
Prasanna Iyer
Product Specialist
Palo Alto Networks
live.paloaltonetworks.com/t5/cloud-ngfw-help-center/ct-p/Cloud_NGFW

L2 Linker

Thanks Prasanna. Regarding Issue 4, can you please update the docs with this info? Currently, neither Github nor Terraform provider docs page state this. 

 

https://github.com/PaloAltoNetworks/terraform-provider-cloudngfwaws/blob/main/docs/index.md

https://registry.terraform.io/providers/PaloAltoNetworks/cloudngfwaws/latest/docs

 

However, your suggestion is valid, because the code seems to be looking for hyphens

https://github.com/PaloAltoNetworks/terraform-provider-cloudngfwaws/blob/main/internal/provider/prov...

 

SZanpure_0-1657195206481.png

 

Also, did you get a chance to get feedback from the backend team about the VPC Endpoint error I am facing above?

 

Thanks,

Shreyas

Hello @SZanpure ,

Greetings!

Regarding your VPC error:
The team has noticed in the documentation that a note mentions that cloudngfwaws_commit_rulestack resource should be placed in a separate plan as the plan that configures the rulestack and its contents. We suggest you put it in a separate plan. 

Kindly refer to this link 
https://registry.terraform.io/providers/PaloAltoNetworks/cloudngfwaws/latest/docs/resources/commit_r... 

Regards,
Prasanna Iyer
Product Specialist
Palo Alto Networks
https://live.paloaltonetworks.com/t5/cloud-ngfw-help-center/ct-p/Cloud_NGFW 


Hi @SZanpure,

I wanted to follow up with you to know if you still need any assistance on the issue.

Regards
Prasanna Iyer
Product Specialist
Palo Alto Networks
https://live.paloaltonetworks.com/t5/cloud-ngfw-help-center/ct-p/Cloud_NGFW 
*Don’t forget to accept the solution provided!*

Hi Prasanna,

Sorry for the late response, but we have decided not to pursue with NGFW for now, since it still feels like a product in the works. I am happy for you to close this issue.

Thanks
Shreyas
  • 4777 Views
  • 8 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!