topic Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin in Cloud NGFW for AWS Discussions

Terraform NGFW provider failing to get token for CloudFirewallAdmin

SZanpure — Thu, 28 Apr 2022 13:32:11 GMT

Hi,

I am trialling out the TF provider in this repo and I have successfully built the provider locally. I am able to configure it as per the settings mentioned in the doc. To give a brief overview, I have

Subscribed to Palo NGFW in AWS Marketplace
Added our sandbox AWS Account to Palo NGFW and run the Cloudformation template which creates cross-account IAM roles.
Enabled Programmatic access and created the IAM role mentioned in the docs to grant access to API Gateway. I have also tagged the IAM role with
Key=NGFWaasRole, Value=CloudFirewallAdmin

Key=NGFWaasRole, Value=CloudRulestackAdmin
tags as mentioned in the other ticket
I have granted sts:AssumeRole permission so that any authenticated user in my sandbox account can assume the above role.
After setting up all of this, I supply the following config values to the provider

host: "api.us-east-1.aws.cloudngfw.paloaltonetworks.com"

region: "us-east-1"

arn: "<The arn of the API Gateway IAM Role I setup in step 3>"

The provider initialises successfully when I run terraform init, however when I run terraform plan, it errors as per the screenshot below

I can only speculate (since I am not familiar with golang, but the code for ngfw client is here ) that the client is failing to execute steps 8 and 9 mentioned here

Another thing to note is that the tags mentioned in step 5 of the above article are different from the ones you have mentioned in the linked ticket above. Is there any reason for this difference? Also, the Github repo linked in step 6 has a broken link, so I cannot view the CFT examples.

Any help would be appreciated since I am now effectively blocked in automating the NGFW firewall creation.

Regards,

Shreyas

Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin

gfreeman — Thu, 28 Apr 2022 17:41:19 GMT

Run terraform apply with logging enabled, it might have more information:

TF_LOG=debug terraform apply

Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin

gfreeman — Thu, 28 Apr 2022 18:43:59 GMT

I have been told that the docs are wrong with regards to the tags needed for the role created in AWS. Change the tags as follows:

CloudFirewallAdmin > CloudNGFWFirewallAdmin
CloudRulestackAdmin > CloudNGFWRulestackAdmin

Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin

SZanpure — Thu, 28 Apr 2022 20:31:41 GMT

Thanks @gfreeman

I made changes to the tags, and it made no difference. After switching on TF_LOG, I got an error message which might be of interest to you.

-----------------------------------------------------: timestamp=2022-04-28T21:05:56.317+0100 2022-04-28T21:05:56.381+0100 [INFO] provider.terraform-provider-cloudngfwaws_v1.0.0: 2022/04/28 21:05:56 [DEBUG] CloudNgfwAws API Response Details: ---[ RESPONSE ]-------------------------------------- HTTP/1.1 403 Forbidden Connection: close Content-Length: 251 Access-Control-Allow-Headers: Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token,X-Amz-User-Agent Access-Control-Allow-Methods: * Access-Control-Allow-Origin: * Content-Type: application/json Date: Thu, 28 Apr 2022 20:05:56 GMT X-Amz-Apigw-Id: <Redacted> X-Amzn-Errortype: AccessDeniedException X-Amzn-Requestid: <Redacted> X-Amzn-Trace-Id: <Redacted> { "message": "User: arn:aws:sts::<Redacted>:assumed-role/palo-ngfw-admin-role/sdk_session is not authorized to perform: execute-api:Invoke on resource: arn:aws:execute-api:us-east-1:********1261:63kpcf9k68/prod/GET/v1/mgmt/tokens/cloudrulestackadmin" }

The palo-ngfw-admin-role is setup as such

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "execute-api:Invoke" ], "Resource": [ "arn:aws:execute-api:*:*:*" ] } ] }

The assume role policy for assuming the role above is set up as such

{ "Version": "2012-10-17", "Statement": [ { "Sid": "AssumeTaggedRole", "Effect": "Allow", "Action": "sts:AssumeRole", "Principal": { "AWS": "arn:aws:iam::${account-id}:root" }, "Condition": { "StringEquals": {"iam:ResourceTag/CloudFirewallAdmin": "CloudNGFWFirewallAdmin"} } } ] }

The IAM role is set up as

resource "aws_iam_role" "palo-ngfw-admin-role" { name = "palo-ngfw-admin-role" assume_role_policy = data.template_file.assume-role-policy.rendered tags = { "CloudFirewallAdmin" = "CloudNGFWFirewallAdmin", "CloudRulestackAdmin" = "CloudNGFWRulestackAdmin" } } resource "aws_iam_policy" "palo-nfgw-admin-policy" { name = "palo-nfgw-admin-policy" policy = data.template_file.palo-ngfw-admin-policy.rendered tags = { "Name" = "palo-nfgw-admin-policy" } } resource "aws_iam_role_policy_attachment" "palo-nfgw-admin-policy-to-palo-ngfw-admin-role" { role = aws_iam_role.palo-ngfw-admin-role.name policy_arn = aws_iam_policy.palo-nfgw-admin-policy.arn }

The provider config is setup as follows

provider "cloudngfwaws" { json_config_file = "./cloudngfwaws_config.json" } { "host": "api.us-east-1.aws.cloudngfw.paloaltonetworks.com", "region": "us-east-1", "arn": "arn:aws:iam::<redacted>:role/palo-ngfw-admin-role" }

After all this set up, the IAM user that I use to login to our sandbox account assumes the palo-ngfw-admin-role and tries to execute the api gateway which subsequently fails. Any ideas please?

Regards,

Shreyas

Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin

gfreeman — Fri, 29 Apr 2022 04:47:28 GMT

Looking at that response, it seems like something is misconfigured with your AWS setup.

Found some other documentation and this does actually have the updated role tags that I mentioned above, so maybe there's something in here that's the answer..?

https://pan.dev/cloudngfw/aws/api

Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin

SZanpure — Fri, 29 Apr 2022 09:36:44 GMT

Thanks @gfreeman

That document is exactly the same as this one. The interesting part is that I had set up the tags properly as per that doc, but the result remains the same. Are you sure the perms on the API Gateway in your AWS Account are configured correctly? It might be that you need to re-deploy the API Gateway if you have changed the policies recently on your side. More info here and here

Here is the IAM Role with the tags

resource "aws_iam_role" "palo-ngfw-admin-role" { name = "palo-ngfw-admin-role" assume_role_policy = data.template_file.assume-role-policy.rendered tags = { "CloudNGFWFirewallAdmin" = "Create and manage firewalls.", "CloudNGFWRulestackAdmin" = "Create and manage local rulestacks.", "CloudNGFWGlobalRulestackAdmin" = "Create and manage global rulestacks." } }

Here is the assume role policy for cloud-firewall-admin

And the IAM policy is configured exactly as per the doc

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "execute-api:Invoke" ], "Resource": [ "arn:aws:execute-api:*:*:*" ] } ] }

From the terraform debug logs, I can see that the role above is being assumed by my IAM user. The NGFW provider API requests are below

2022-04-29T09:45:12.557+0100 [DEBUG] provider: starting plugin: path=.terraform/providers/terraform.local/local/cloudngfwaws/1.0.0/darwin_amd64/terraform-provider-cloudngfwaws_v1.0.0 args=[.terraform/providers/terraform.local/local/cloudngfwaws/1.0.0/darwin_amd64/terraform-provider-cloudngfwaws_v1.0.0] 2022-04-29T09:45:12.562+0100 [DEBUG] provider: plugin started: path=.terraform/providers/terraform.local/local/cloudngfwaws/1.0.0/darwin_amd64/terraform-provider-cloudngfwaws_v1.0.0 pid=43578 2022-04-29T09:45:12.562+0100 [DEBUG] provider: waiting for RPC address: path=.terraform/providers/terraform.local/local/cloudngfwaws/1.0.0/darwin_amd64/terraform-provider-cloudngfwaws_v1.0.0 2022-04-29T09:45:12.575+0100 [INFO] provider.terraform-provider-cloudngfwaws_v1.0.0: configuring server automatic mTLS: timestamp=2022-04-29T09:45:12.574+0100 2022-04-29T09:45:12.593+0100 [DEBUG] provider: using plugin: version=5 2022-04-29T09:45:12.593+0100 [DEBUG] provider.terraform-provider-cloudngfwaws_v1.0.0: plugin address: address=/var/folders/xb/lm10djv969x0kn2qkt_kpns80000gp/T/plugin580437062 network=unix timestamp=2022-04-29T09:45:12.593+0100 2022-04-29T09:45:12.626+0100 [WARN] ValidateProviderConfig from "provider[\"registry.terraform.io/hashicorp/aws\"]" changed the config value, but that value is unused 2022-04-29T09:45:12.630+0100 [DEBUG] No provider meta schema returned 2022-04-29T09:45:12.632+0100 [DEBUG] provider.terraform-provider-aws_v4.10.0_x5: Using profile from configuration: "sandbox-networking": timestamp=2022-04-29T09:45:12.631+0100 2022-04-29T09:45:12.632+0100 [INFO] provider.terraform-provider-aws_v4.10.0_x5: Retrieved credentials from "SharedConfigCredentials: /Users/Shreyas.Zanpure/.aws/credentials": timestamp=2022-04-29T09:45:12.632+0100 2022-04-29T09:45:12.632+0100 [DEBUG] provider.terraform-provider-aws_v4.10.0_x5: Trying to get account information via sts:GetCallerIdentity: timestamp=2022-04-29T09:45:12.632+0100 2022-04-29T09:45:12.633+0100 [INFO] ReferenceTransformer: reference not found: "path.module" 2022-04-29T09:45:12.633+0100 [DEBUG] ReferenceTransformer: "module.base.data.template_file.init" references: [] 2022-04-29T09:45:12.633+0100 [DEBUG] provider.terraform-provider-aws_v4.10.0_x5: [aws-sdk-go-v2] Request POST / HTTP/1.1 Host: sts.us-east-1.amazonaws.com User-Agent: APN/1.0 HashiCorp/1.0 Terraform/1.0.7 (+https://www.terraform.io) terraform-provider-aws/dev (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go-v2/1.16.2 os/macos lang/go/1.17.6 md/GOOS/darwin md/GOARCH/amd64 api/sts/1.16.0 Content-Length: 43 Amz-Sdk-Invocation-Id: 8334faa7-d169-43e6-b42f-c4029b5ce990 Amz-Sdk-Request: attempt=1; max=25 Authorization: AWS4-HMAC-SHA256 Credential=AKIAWZZ3MAPME2WKYNVE/20220429/us-east-1/sts/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-date, Signature=624cd938ef86b60469f73a6219155b8a76b6bdf77b10d6c16e14d566b0326025 Content-Type: application/x-www-form-urlencoded X-Amz-Date: 20220429T084512Z Accept-Encoding: gzip Action=GetCallerIdentity&Version=2011-06-15: timestamp=2022-04-29T09:45:12.633+0100 2022-04-29T09:45:12.633+0100 [INFO] ReferenceTransformer: reference not found: "path.module" 2022-04-29T09:45:12.633+0100 [INFO] ReferenceTransformer: reference not found: "var.account-id" 2022-04-29T09:45:12.633+0100 [DEBUG] ReferenceTransformer: "module.base.data.template_file.assume-role-policy" references: [] 2022-04-29T09:45:12.633+0100 [INFO] ReferenceTransformer: reference not found: "path.module" 2022-04-29T09:45:12.633+0100 [DEBUG] ReferenceTransformer: "module.base.data.template_file.palo-ngfw-admin-policy" references: [] 2022-04-29T09:45:12.636+0100 [INFO] provider.terraform-provider-cloudngfwaws_v1.0.0: 2022/04/29 09:45:12 (login) refreshing JWTs...: timestamp=2022-04-29T09:45:12.636+0100 2022-04-29T09:45:12.636+0100 [INFO] provider.terraform-provider-cloudngfwaws_v1.0.0: 2022/04/29 09:45:12 (login) refreshing rulestack JWT...: timestamp=2022-04-29T09:45:12.636+0100 2022-04-29T09:45:12.637+0100 [INFO] provider.terraform-provider-cloudngfwaws_v1.0.0: 2022/04/29 09:45:12 (login) refreshing firewall JWT...: timestamp=2022-04-29T09:45:12.637+0100 2022-04-29T09:45:12.637+0100 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/template/2.2.0/darwin_amd64/terraform-provider-template_v2.2.0_x4 pid=43577 2022-04-29T09:45:12.637+0100 [DEBUG] provider: plugin exited 2022-04-29T09:45:13.007+0100 [INFO] provider.terraform-provider-cloudngfwaws_v1.0.0: 2022/04/29 09:45:13 [DEBUG] CloudNgfwAws API Request Details: ---[ REQUEST ]--------------------------------------- GET /v1/mgmt/tokens/cloudrulestackadmin HTTP/1.1 Host: api.us-east-1.aws.cloudngfw.paloaltonetworks.com User-Agent: Terraform/1.0.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.10.1 terraform-provider-cloudngfwaws/dev Content-Length: 65 Authorization: AWS4-HMAC-SHA256 Credential=ASIAWZZ3MAPMJZ4GVKG6/20220429/us-east-1/execute-api/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-security-token, Signature=e13e27cac4a60e4b93e1948cda678f21a2831a12b81d7c0bbbc8aac4cbdae8c7 Content-Type: application/json X-Amz-Date: 20220429T084513Z X-Amz-Security-Token: FwoGZXIvYXdzEEIaDHY/DJ8ESp2CSON+3iKvAbnQ+A3+se4jEjoXRmzrYU1olbKNc+ABgEKeNMXKwWLvZ4+YXb3Omorva6Y+E9RkJXOvbCT7iuY2N9nxu/pNri1E7pAGEchHkOQ4slk2C0K2vPipEVg3eKyUxtch7WynS51SeY04RP4qZipcF/UPzTM2OPo3EFL3lWqDH314tkIUdXVZ9UUidvabu/dhzyRUh+OeJaZCu1KPBdB5xoZr1mKz5DAVZcsQjaQXbShCH/comcuukwYyLaCDABerF3X0zuEAEnNHpDZXMbvrHHI8/MFXtA41UTFHmDrKsjkWiacLv/s86g== Accept-Encoding: gzip { "ExpiryTime": 120, "KeyInfo": { "Region": "us-east-1", "Tenant": "XY" } } -----------------------------------------------------: timestamp=2022-04-29T09:45:13.006+0100 2022-04-29T09:45:13.013+0100 [INFO] provider.terraform-provider-cloudngfwaws_v1.0.0: 2022/04/29 09:45:13 [DEBUG] CloudNgfwAws API Request Details: ---[ REQUEST ]--------------------------------------- GET /v1/mgmt/tokens/cloudfirewalladmin HTTP/1.1 Host: api.us-east-1.aws.cloudngfw.paloaltonetworks.com User-Agent: Terraform/1.0.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.10.1 terraform-provider-cloudngfwaws/dev Content-Length: 65 Authorization: AWS4-HMAC-SHA256 Credential=ASIAWZZ3MAPMAVXHJH5M/20220429/us-east-1/execute-api/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-security-token, Signature=c25896b944eb1d50da387e0b28ad5e80c5a582f2baa1f8c580bacdf2f1a93bba Content-Type: application/json X-Amz-Date: 20220429T084513Z X-Amz-Security-Token: FwoGZXIvYXdzEEIaDNTbGd35gptfaUWX+yKvARSKRrecP/LfUW+zY3HdQ2K/VdipSABiLgEP9o/tzb+MF2U/LxZ0AsSfl24+54TTbnVils9715vgwC27Pz+tc5rpF/gGHRkPH6Mh+iN0wio1h7Ts3mhfvhUAdX22+sgB1L6uSfLVX3+t/maUVSPqnORoZRJ6Bzk7c+0nb9h0FHWogvQXoIIp/6gWvoObSxAwUf+LBU3vDM6mZI2/pmkseJadGox7lYHT/gzmljW/7+komcuukwYyLQs7Qtr30KYv9mgPOi4eIszwncrby1xF0QblenvW1dpTgvEdIwiSjBzmVuf2xA== Accept-Encoding: gzip { "ExpiryTime": 120, "KeyInfo": { "Region": "us-east-1", "Tenant": "XY" } }

As you can see, the provider tries to refresh JWT for both firewalladmin and rulestackadmin. The responses are below

2022-04-29T09:45:13.452+0100 [INFO] provider.terraform-provider-cloudngfwaws_v1.0.0: 2022/04/29 09:45:13 [DEBUG] CloudNgfwAws API Response Details: ---[ RESPONSE ]-------------------------------------- HTTP/1.1 403 Forbidden Connection: close Content-Length: 250 Access-Control-Allow-Headers: Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token,X-Amz-User-Agent Access-Control-Allow-Methods: * Access-Control-Allow-Origin: * Content-Type: application/json Date: Fri, 29 Apr 2022 08:45:13 GMT X-Amz-Apigw-Id: RVbQAHLYoAMF4eg= X-Amzn-Errortype: AccessDeniedException X-Amzn-Requestid: 2f112342-1b79-4ed9-9f47-5dc322991474 X-Amzn-Trace-Id: Root=1-626ba599-385c55af49a880b737155da7 { "message": "User: arn:aws:sts::467739083736:assumed-role/palo-ngfw-admin-role/sdk_session is not authorized to perform: execute-api:Invoke on resource: arn:aws:execute-api:us-east-1:********1261:63kpcf9k68/prod/GET/v1/mgmt/tokens/cloudfirewalladmin" } -----------------------------------------------------: timestamp=2022-04-29T09:45:13.452+0100 2022-04-29T09:45:13.453+0100 [INFO] provider.terraform-provider-cloudngfwaws_v1.0.0: 2022/04/29 09:45:13 [DEBUG] CloudNgfwAws API Response Details: ---[ RESPONSE ]-------------------------------------- HTTP/1.1 403 Forbidden Connection: close Content-Length: 251 Access-Control-Allow-Headers: Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token,X-Amz-User-Agent Access-Control-Allow-Methods: * Access-Control-Allow-Origin: * Content-Type: application/json Date: Fri, 29 Apr 2022 08:45:13 GMT X-Amz-Apigw-Id: RVbQAEDdoAMFlrw= X-Amzn-Errortype: AccessDeniedException X-Amzn-Requestid: 0544aee5-fcad-4bdd-8409-1017e4c4690e X-Amzn-Trace-Id: Root=1-626ba599-539f933c60fb6d616b9c83b9 { "message": "User: arn:aws:sts::467739083736:assumed-role/palo-ngfw-admin-role/sdk_session is not authorized to perform: execute-api:Invoke on resource: arn:aws:execute-api:us-east-1:********1261:63kpcf9k68/prod/GET/v1/mgmt/tokens/cloudrulestackadmin" }

At this point I am not really sure what is mis-configured on my side. I have followed the docs as mentioned. Also the link for this repo https://github.com/PaloAltoNetworks/CloudNGFW-AWS-Examples seems to be broken (mentioned in step 6 of the guide I linked at the beginning of this post), so I cannot look at the CFT examples either.

Regards,

Shreyas

Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin

gfreeman — Fri, 29 Apr 2022 16:24:41 GMT

OK, two things:

First is that I think the value of the tag is the important bit, not the key name. In my setup, the keys are actually named "CloudNGFWFirewallAdmin1" and "CloudNgfwRulestackAdmin". Try setting the values to the values mentioned in the 2nd link I provided and see if that changes the result.

Second, since you are essentially stuck at, "I'm trying to enable API access," this is fully supported TAC territory. If you have a support contract with Palo Alto Networks in place, then I'd encourage you to reach out to TAC for help. The Terraform provider is unreleased currently and should otherwise be considered beta code, but enabling API access has nothing to do with the provider.

Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin

gfreeman — Fri, 29 Apr 2022 16:42:15 GMT

Oh sorry, another thing: I just noticed that in your error message, the ARN looks different from mine, but there's enough stuff going on that I don't know if this is a result of how the AWS SDK formats things or not. And looking back, it does seem like you're using the ARN of the IAM role, but I just wanted to make sure... The ARN that I specify in the provider configure block looks like this, just wanted to be certain that's what yours looks like also (I was using two different ARNs instead of a single one):

provider "cloudngfwaws" { lfa_arn = "arn:aws:iam::<number>:role/CloudNGFWFirewallAdmin1" lra_arn = "arn:aws:iam::<number>:role/CloudNgfwRuleStackAdmin" #.... other params }

Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin

SZanpure — Fri, 29 Apr 2022 16:56:04 GMT

NBS Internal

Hi,

Would it be possible for you to supply the exact values of the “tag KV pairs” for the IAM role please? I have put the exact key/values for the tags from the article you linked to, where the value seems to be a sort of a description of the key. It would help clarify things a bit.

We do have a support contract as part of “Nationwide Building Society”, so I will try and find someone who can help me under that.

Thanks,
Shreyas.

Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin

gfreeman — Fri, 29 Apr 2022 18:22:36 GMT

I can't share my config, unfortunately..... But I'm glad that you can engage TAC for figuring out API access.

Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin

SZanpure — Tue, 03 May 2022 10:51:21 GMT

It looks like the docs have been updated now along with provider (as you mentioned) with the correct tags and steps. I will try these out now and report back. Thanks 🙂

EDIT: I have got it working now. There is one another tweak that needs to be done. The KeyInfo property on line 14 in getJwtStruct needs to be commented out

https://github.com/PaloAltoNetworks/cloud-ngfw-aws-go/blob/main/auth.go

This is because the commit made here commented out the code supplying the property value. After I commented out the property locally, I rebuilt the provider, and I was able to run 'terraform plan' successfully 🙂

The error I was getting without commenting out KeyInfo was

ValidationError: 1 validation error for GetCloudFirewallAdminRequest KeyInfo extra fields not permitted (type=value_error.extra)

After commenting out, I got successful JWT generation and terraform plan out as follows

2022-05-03T11:37:08.302+0100 [INFO] provider.terraform-provider-cloudngfwaws_v1.0.0: 2022/05/03 11:37:08 sending: {"ExpiryTime":120}: timestamp=2022-05-03T11:37:08.302+0100 2022-05-03T11:37:08.302+0100 [INFO] provider.terraform-provider-cloudngfwaws_v1.0.0: 2022/05/03 11:37:08 path: https://api.us-east-1.aws.cloudngfw.paloaltonetworks.com/v1/mgmt/tokens/cloudfirewalladmin: timestamp=2022-05-03T11:37:08.302+0100 2022-05-03T11:37:08.302+0100 [INFO] provider.terraform-provider-cloudngfwaws_v1.0.0: 2022/05/03 11:37:08 [DEBUG] CloudNgfwAws API Request Details: ---[ REQUEST ]--------------------------------------- GET /v1/mgmt/tokens/cloudfirewalladmin HTTP/1.1 Host: api.us-east-1.aws.cloudngfw.paloaltonetworks.com User-Agent: Terraform/1.0.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.10.1 terraform-provider-cloudngfwaws/dev Content-Length: 18 Authorization: AWS4-HMAC-SHA256 Credential=<redacted>/20220503/us-east-1/execute-api/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-security-token, Signature=<redacted> Content-Type: application/json X-Amz-Date: 20220503T103708Z X-Amz-Security-Token: <redacted> Accept-Encoding: gzip { "ExpiryTime": 120 } 2022-05-03T11:37:11.650+0100 [INFO] provider.terraform-provider-cloudngfwaws_v1.0.0: 2022/05/03 11:37:11 [DEBUG] CloudNgfwAws API Response Details: ---[ RESPONSE ]-------------------------------------- HTTP/1.1 200 OK Connection: close Content-Length: 874 Access-Control-Allow-Origin: * Cache-Control: no-store Content-Security-Policy: frame-ancestors none Content-Type: application/json Date: Tue, 03 May 2022 10:37:11 GMT Strict-Transport-Security: max-age=31536000 X-Amz-Apigw-Id: <redacted> X-Amzn-Requestid: 12db701d-62b9-4097-9a1b-3a8d509c86f2 X-Amzn-Trace-Id: Root=1-627105d4-122e8dba707e6988101c1aa8 X-Content-Type-Options: no-sniff X-Frame-Options: DENY { "Response": { "TokenId": <redacted>, "SubscriptionKey": <redacted>, "ExpiryTime": 120, "Enabled": true }, "ResponseStatus": { "ErrorCode": 0 } }

Terraform will perform the following actions: # module.base.cloudngfwaws_commit_rulestack.example will be created + resource "cloudngfwaws_commit_rulestack" "example" { + commit_errors = (known after apply) + commit_status = (known after apply) + id = (known after apply) + rulestack = "test-rulestack-terraform" + state = "Running" + validation_errors = (known after apply) + validation_status = (known after apply) } # module.base.cloudngfwaws_ngfw.test-ngfw-terraform will be created + resource "cloudngfwaws_ngfw" "test-ngfw-terraform" { + account_id = "467739083736" + app_id_version = (known after apply) + automatic_upgrade_app_id_version = true + description = "Test NGFW created via terraform provider" + endpoint_mode = "CustomerManaged" + id = (known after apply) + name = "test-ngfw-terraform" + rulestack = "test-rulestack-terraform" + tags = { + "Foo" = "bar" } + update_token = (known after apply) + vpc_id = "vpc-0f9f37d5ba25bf53e" + subnet_mapping { + subnet_id = "subnet-063a57e75839ba2ba" } + subnet_mapping { + subnet_id = "subnet-016286207f6cede42" } } # module.base.cloudngfwaws_rulestack.test-rulestack-terraform will be created + resource "cloudngfwaws_rulestack" "test-rulestack-terraform" { + account_id = "467739083736" + description = "Test rulestack created via terraform provider" + id = (known after apply) + minimum_app_id_version = (known after apply) + name = "test-rulestack-terraform" + scope = "Local" + state = (known after apply) + profile_config { + anti_spyware = "BestPractice" + anti_virus = "BestPractice" + file_blocking = "BestPractice" + url_filtering = "None" + vulnerability = "BestPractice" } } # module.base.cloudngfwaws_security_rule.test-security-rule-terraform will be created + resource "cloudngfwaws_security_rule" "test-security-rule-terraform" { + action = "Allow" + applications = [ + "any", ] + audit_comment = "initial config" + description = "Test security rule created via terraform provider" + enabled = true + id = (known after apply) + logging = true + name = "test-security-rule-terraform" + negate_destination = true + priority = 1 + protocol = "application-default" + rule_list = "LocalRule" + rulestack = "test-rulestack-terraform" + tags = (known after apply) + update_token = (known after apply) + category {} + destination { + cidrs = [ + "any", ] } + source { + cidrs = [ + "any", ] } } Plan: 4 to add, 0 to change, 0 to destroy.

Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin

gfreeman — Tue, 03 May 2022 16:57:08 GMT

I keep forgetting how golang treats XML and JSON is not the same. Ok, this should be fixed now, just re-pull and SDK and the provider. I added NGFW tagging and rulestack tagging last night.

Awesome to hear it's working for you!

Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin

SZanpure — Tue, 03 May 2022 18:46:07 GMT

Thanks 🙂

I did get stuck on another thing though. The rulestack resource execution was followed by commit_rulestack resource. The default state is set to Running. However, when the NFGW resource was being created, I had a timeout after around 5 minutes and 30 seconds. When I ran tf plan once more, tf wanted to change state from “precommitdone” to “running” (due to default state in terraform resource block being “running” and state on Palo NGFW UI being “precommitdone”. I wasn’t able to run terraform apply again due to recurring error

module.base.cloudngfwaws_commit_rulestack.commit-test-rulestack-terraform: Creating... ╷ │ Error: Error(1): Commit process unfinished. │ │ with module.base.cloudngfwaws_commit_rulestack.commit-test-rulestack-terraform, │ on modules/base/palo-ngfw.tf line 33, in resource "cloudngfwaws_commit_rulestack" "commit-test-rulestack-terraform": │ 33: resource "cloudngfwaws_commit_rulestack" "commit-test-rulestack-terraform" {

I am also unable to delete the rulestack from NGFW UI.

Have you encountered this? The docs suggest that I should look at the read-only properties on the rulestack to find it’s status, which can be done via output vars, but does mean that all ruelstacks need to be defined in their own module?

It would be great if the docs could include an example of some sort dependency, where NGFW is created only after rulestack has switched to a status of “Running” in Palo UI, instead of setting the default state in terraform which does not truly reflect the state in Palo UI. Hope that makes sense.

Thanks,

Shreyas

Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin

gfreeman — Wed, 04 May 2022 16:16:31 GMT

You shouldn't have cloudngfwaws_commit_rulestack in the same plan file as cloudngfwaws_rulestack, this is called out in the docs. This is because of the order of operations that Terraform does things in, there will always be, what is referred to in Terraform, as "configuration drift." Commit rulestack needs to be in its own plan file, but can be paired with other commit rulestack operations.

The commit rulestack resource has 15min set as the timeout, so I don't know why it timed out after 5min...? I've also not encountered PrecommitDone sticking around for a long time, but I suspect at this point it is no longer in this state..?

Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin

gfreeman — Wed, 04 May 2022 16:22:09 GMT

Actually, I've only seen SUCCESS / FAILURE / PENDING, I've not seen PrecommitDone before. If you do terraform refresh / terraform show, is it showing that the cloudngfwaws_commit_rulestack.(terraform_name).CommitStatus is PrecommitDone...??

Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin

SZanpure — Wed, 04 May 2022 17:16:33 GMT

NBS Internal

Thanks for the info. I have run terraform plan multiple times and the rulestack commit status is “PreCommitDone”. I am unable to delete it from the UI since it gives that error in the previously posted screenshot. I will look to moving commit_rulestack to separate plan, which means that NGFW resource will need to be in its own plan as well? It has a dependency on the rullestack being committed before the NGFW is successfully deployed.

Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin

gfreeman — Wed, 04 May 2022 17:22:47 GMT

...huh. Is the rulestack associated with a NGFW which you've spun up? If so, you might have to bring down / delete that NGFW before you can delete the rulestack.

Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin

SZanpure — Wed, 04 May 2022 17:51:39 GMT

NBS Internal

Yes. If you look at the terraform plan output from previous posts, I am attempting creation of barebones firewall, with a rulestack that has a single security rule.

So, in my head I understand the execution sequence as

1. Rule is created
2. Rulestack is created with the above rule
3. Rulestack needs to be committed.
4. Rulestack is associated with the NGFW
5. NGFW is created.

In my case, I believe I am stuck at step 3. Rulestack is not properly committed, and hence the first time around, NGFW creation in “tf apply” ran for 5 minutes and 30 seconds before timing out. Now I am not sure if “tf apply” on NGFW was waiting internally on Commit_Rulestack to finish or it was waiting for its own creation.

Another thing to note is that in the UI my NGFW via terraform appeared in CREATE_FAIL status (after 5.5 minutes timeout).
I was able to delete the firewall in the UI, but unable to delete the rule stack.

I will investigate creation of separate tfstate/plan for rulestack and commit_rulestack.

Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin

SZanpure — Thu, 05 May 2022 15:57:24 GMT

@gfreeman The way in which resources get created "outside" terraform is not helping at all. I was able to create rulestack and firewall , yet when I try to create vpc endpoints on my side using vpc endpoint service for the NGFW, I get

│ Error: Error(1): Commit process unfinished.

In the UI, everything looks ok

Below is my terraform code

data "cloudngfwaws_ngfw" "palo-ngfw" { name = var.ngfw-name } data "aws_vpc_endpoint_service" "palo-ngfw-vpce-service" { service_name = data.cloudngfwaws_ngfw.palo-ngfw.endpoint_service_name } resource "aws_vpc_endpoint" "palo-ngfw-vpce-private-subnet-1" { vpc_id = var.inspection-vpc-id service_name = data.aws_vpc_endpoint_service.palo-ngfw-vpce-service.service_name vpc_endpoint_type = data.aws_vpc_endpoint_service.palo-ngfw-vpce-service.service_type subnet_ids = [var.inspection-vpc-private-subnet-1-id] private_dns_enabled = false tags = { "Name" = "palo-ngfw-vpce-private-subnet-1" } } resource "aws_vpc_endpoint" "palo-ngfw-vpce-private-subnet-2" { vpc_id = var.inspection-vpc-id service_name = data.aws_vpc_endpoint_service.palo-ngfw-vpce-service.service_name vpc_endpoint_type = data.aws_vpc_endpoint_service.palo-ngfw-vpce-service.service_type subnet_ids = [var.inspection-vpc-private-subnet-2-id] private_dns_enabled = false tags = { "Name" = "palo-ngfw-vpce-private-subnet-2" } } resource "aws_route" "inspection-rtb-tgw-subnet-1" { route_table_id = var.inspection-vpc-tgw-rtb-subnet-1-id destination_cidr_block = "0.0.0.0/0" vpc_endpoint_id = aws_vpc_endpoint.palo-ngfw-vpce-private-subnet-1.id } resource "aws_route" "inspection-rtb-tgw-subnet-2" { route_table_id = var.inspection-vpc-tgw-rtb-subnet-2-id destination_cidr_block = "0.0.0.0/0" vpc_endpoint_id = aws_vpc_endpoint.palo-ngfw-vpce-private-subnet-2.id }

What am I missing?

Re: Terraform NGFW provider failing to get token for CloudFirewallAdmin

gfreeman — Mon, 09 May 2022 16:42:57 GMT

This feels like TAC territory again to me... You are in a situation where even using the GUI is not working / allowing you to recover.

The AWS config I am honestly not very familiar with, so I won't be able to provide useful feedback on that part of it, sorry..

But to double back on what I was saying before: you'll want to have 3 separate directories for your Terraform config:

dir1) All the code that defines both the rulestack and its contents (prefix lists, intelligent feeds, etc)

dir2) cloudngfwaws_commit_rulestack only

dir3) All your cloudngfwaws_ngfw instances

-- Edit --

Forgot to mention: and you only run the Dir3 terraform code if the commit shows as successfully committed from Dir2 (check using terraform show). Currently the cloudngfwaws_commit_rulestack only returns an error if issuing the commit command fails, not if the commit itself fails. It's possible that this will change before the provider exits beta, but this is the current implementation.