Deploy Prisma Cloud Compute Console in Kubernetes behind NGINX Ingress

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
L1 Bithead
100% helpful (1/1)

By Juan Montufar, Senior Customer Success Engineer

and

Sriram Choudary Nimmagadda, Senior Customer Success Engineer

 

Introduction 

This document showcases the process of how to deploy the Prisma Cloud Compute console in a Kubernetes cluster on any cloud provider and use a NGINX Ingress controller as a proxy for this console.

Purpose
For many enterprises, moving production workloads into Kubernetes brings additional challenges and complexities around application traffic management. An Ingress controller abstracts away the complexity of Kubernetes application traffic routing and provides a bridge between Kubernetes services and external ones.

 

What’s Next?
You’ll see how to deploy the Prisma Cloud Compute console behind an NGINX ingress using Mutual Authentication by either reusing the certificates that the console provides for defender - consolo communication, or using your own custom certificates.

 

Ingress deployment

​​

Objective

Discover how to deploy an Ingress controller to manage the Web UI or API access to the Prisma Cloud Compute console and the defender-console communication.

 

Environment

  • A Kubernetes Cluster on any public or private cloud.
  • A Domain on any internet domain name registrar like AWS Route 53, Google Domains, Cloudflare, Godaddy, Namecheap, or others. 

 

Requirements

This document requires you to have the following tools in place: 

  1. Helm and kubectl installed on your working machine.
  2. Access to your cluster using the kubectl and helm tools.
  3. Openssl installed.
  4. Ability to create DNS Records type A and CNAME in your domain.
  5. Default storageClass in your kubernetes cluster


Part 1: Setting up the Console

 

Step 1: Install NGINX Ingress controller

Run the following command:

$ helm upgrade --install ingress-nginx ingress-nginx \
  --repo https://kubernetes.github.io/ingress-nginx \
  --namespace ingress-nginx --create-namespace

It's recommended to follow your cloud provider’s instructions on how to deploy a nginx ingress resource.

 

Step 2: Create DNS Records

You need to obtain the public IP address or common name of the ingress service:

$ kubectl get svc -n ingress-nginx

 

NAME                     TYPE         CLUSTER-IP   EXTERNAL-IP     PORT(S)          AGE
Ingress-nginx-controller LoadBalancer xx.xx.xx.xx  PUBLIC_IP  80:xx/TCP,443:xx/TCP   44h

Figure 1: kubectl get svc command output_palo-alto-networks

 

Create two DNS records, one for accessing the Prisma Cloud Compute console web UI. In this document, the variable is named CONSOLE_FQDN which is “console.example.com”. Another for the defender - console communication is named as DEFENDER_FQDN which can be like “defender.example.com”. 

 

Step 3: Download the Console Installation Package

Download the appropriate release from the Support Portal and then create the prisma_cloud folder and unpack the release tarball:

$ mkdir pc_VERSION
$ tar -xzvf prisma_cloud_compute_edition_VERSION.tar.gz -C pc_VERSION/
$ cd pc_VERSION/

 

Step 4: Verify Storage Classes

Check if your cluster has any storage class installed by running the following command:

$ kubectl get storageclasses

 

NAME                    PROVISIONER          RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE

default (default)       xxxxxxxxxx           Delete          WaitForFirstConsumer   true                   120d

Figure 2: storage classes installed in the kubernetes cluster_palo-alto-networks

 

As long as your cluster has a storage class we can continue, otherwise you need to install one. We suggest relying on your cloud provider. 

 

Step 5:  Generate a YAML configuration file for Console

$ PLATFORM/twistcli console export kubernetes --service-type ClusterIP

The PLATFORM can be linux, osx or windows. 

You will be asked to enter the product token received as part of the purchase of Prisma Cloud Compute subscription.

In this scenario, we are going to use the default storageClass so it is not required to declare which storage class we are using, but if you want to set up your own storageClass you can use the flag --storage-class YOUR_STORAGE_CLASS_NAME.

 

Step 6: Deploy the Prisma Cloud Console

$ kubectl apply -f twistlock_console.yaml

 

Step 7: Generate the console certificates for Web UI and API access

There are two options for the certificates:

  1.  Use a cert-manager generated secret 
  2. create a tls secret from a certificate and key file

In this scenario, we’ll use the second option:

$ kubectl create secret tls console-tls -n twistlock --cert=console-cert.pem --key=console-key.pem

 

Step 8: Deploy Console Ingress

Create a file console_ingress.yaml with the following content:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: console-ingress
  namespace: twistlock
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
spec:
  rules:
  - host: ${CONSOLE_FQDN}
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: twistlock-console
            port:
              number: 8083
  tls:
  - secretName: console-tls
    hosts:
    - ${CONSOLE_FQDN}

Figure 3: console ingress manifest_palo-alto-networks

 

Apply the file created:

$ export CONSOLE_FQDN=console.example.com
$ envsubst < console_ingress.yaml | kubectl apply -f -

 

Step 9: Create the user and add the license key

In your browser input the following URL:

$ https://CONSOLE_FQDN

You’ll be asked to create an account and input your license key.

Step 10: Add Subject Alternative Name for the defender FQDN

On the Prisma Cloud Compute console, go to Manage > Defenders > Names > Add SAN, and input the value of DEFENDER_FQDN:

 

RPrasadi_3-1697565996675.png

Figure 4: Console Subject Alternative names_palo-alto-networks

 

Step 11: Download the DaemonSet YAML file for the defender

On the Prisma Cloud Compute console, go to Manage > Defenders > Deployed Defenders > Manual Deploy. From the dropdown menu, choose DEFENDER_FQDN, set communication port to 443 and other relevant parameters, and then click on Download YAML.

 

RPrasadi_4-1697565996480.png

Figure 5: Manual deployment of Defenders_palo-alto-networks

 

 

Part 2: Setup defender - console communication

 

Console Certificates Method

 

This method is used if you have access to the console pod via kubectl.

 

Step 1: Copy the console certificates and keys to local directory

Get the twistlock console pod full name:

$ kubectl get pods -n twistlock 

NAME                                       READY   STATUS    RESTARTS   AGE

twistlock-console-xxxxxxxxx-xxxxx   1/1          Running    0                   106m

Figure 6: kubectl get pods command output_palo-alto-networks

 

Copy the files defender-ca.pem, defender-server-cert.pem, defender-server-key.pem, defender-client-cert.pem, defender-server-client.pem and service-parameter found in directory /var/lib/twistlock/certificates to the a local directory.

$ mkdir certs
$ cd certs/
$ POD_NAME=twistlock-console-xxxxxxxxx-xxxxx
$ kubectl cp twistlock/$POD_NAME:/var/lib/twistlock/certificates/defender-ca.pem rootCA.pem
$ kubectl cp twistlock/$POD_NAME:/var/lib/twistlock/certificates/defender-server-cert.pem defender-server-cert.pem 
$ kubectl cp twistlock/$POD_NAME:/var/lib/twistlock/certificates/defender-client-cert.pem defender-client-cert.pem 
$ kubectl cp twistlock/$POD_NAME:/var/lib/twistlock/certificates/defender-server-key.pem encrypted-defender-server-key.pem 
$ kubectl cp twistlock/$POD_NAME:/var/lib/twistlock/certificates/defender-client-key.pem encrypted-defender-client-key.pem
$ kubectl cp twistlock/$POD_NAME:/var/lib/twistlock/certificates/service-parameter service-parameter

 

The files defender-ca.pem, defender-client-cert.pem and defender-client-key.pem can be found in the daemonset.pem file in the secret twistlock-secrets, but are encoded in base 64.

 

Step 2: Decrypt the key files

$ openssl rsa -in encrypted-defender-server-key.pem -out defender-server-key.pem -passin file:service-parameter
$ openssl rsa -in encrypted-defender-client-key.pem -out defender-client-key.pem -passin file:service-parameter

 

Step 3: Generate secrets for ingress 

$ kubectl create secret generic ca-secret -n twistlock --from-file=ca.crt=rootCA.pem
$ kubectl create secret generic proxy-ca-secret -n twistlock --from-file=ca.crt=rootCA.pem --from-file=tls.crt=defender-client-cert.pem --from-file=tls.key=defender-client-key.pem
$ kubectl create secret tls proxy-tls -n twistlock --cert defender-server-cert.pem --key defender-server-key.pem

 

Step 4: Create the Console Defender Ingress

Create a file defender_ingress.yaml with the following content:

 

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: defender-ingress
  namespace: twistlock
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.org/websocket-services: "twistlock-console"
    nginx.ingress.kubernetes.io/auth-tls-match-cn: ${DEFENDER_FQDN}
    nginx.ingress.kubernetes.io/auth-tls-secret: twistlock/ca-secret
    nginx.ingress.kubernetes.io/auth-tls-verify-client: "on"
    nginx.ingress.kubernetes.io/auth-tls-verify-depth: "1"
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
    nginx.ingress.kubernetes.io/proxy-ssl-secret: twistlock/proxy-ca-secret
    nginx.ingress.kubernetes.io/proxy-ssl-verify: "on"
    nginx.ingress.kubernetes.io/proxy-ssl-verify-depth: "1"
    nginx.ingress.kubernetes.io/configuration-snippet: |
      proxy_ssl_name "$DEFENDER_FQDN";
spec:
  rules:
  - host: ${DEFENDER_FQDN}
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: twistlock-console
            port:
              number: 8084
  tls:
  - secretName: proxy-tls
    hosts:
    - ${DEFENDER_FQDN}

Figure 7: defender ingress manifest_palo-alto-networks

 

Apply the file created:

$ export DEFENDER_FQDN=defender.example.com
$ envsubst < defender_ingress.yaml | kubectl apply -f -

 

The ingress created is meant to perform Mutual Authentication between in the frontend and in the backend.

 

Step 5: Deploy the Defender

Apply the modified daemonset.yaml file on different kubernetes context

$ kubectl apply -f daemonset.yaml

 

Custom Certificates Method

This method assumes that you don’t have access to the console pod due to internal policies or admission control, and you have not deployed any defender as of yet. If you have defenders deployed, you’ll need to update the certificates or do a mix-up with the previous method. 

 

Step 1: Create a password to encrypt keys

This step is not required, but recommended. Run the following commands:

$ openssl rand -hex 24 > mypass
$ openssl enc -aes256 -pbkdf2 -salt -in mypass -out mypass.enc

You’ll be asked to input a passphrase. After that, you can remove the file mypass

 

Step 2: Generate a root CA certificate

1. Generate an encrypted root CA key

$ openssl genrsa -des3 -passout file:mypass.enc -out rootCA-key.pem 4096

2. Create and self-sign the root certificate

$ openssl req -x509 -new -nodes -key rootCA-key.pem -sha256 -days 3650 -out rootCA.pem -subj "/OU=TwistlockCustomCert/CN=RootCA" -passin file:mypass.enc

 

Step 3: Generate a defender-server certificate for the console

1. Create SAN Names file

$ CONSOLE_FQDN=console.example.com
$ DEFENDER_FQDN=defender.example.com
$ CLUSTER_IP=$(kubectl get svc twistlock-console  -n twistlock -o jsonpath="{.spec.clusterIP}") 
$ printf "subjectAltName=@alt_names\n\n[alt_names]\nDNS.1=$CONSOLE_FQDN\nDNS.2=$DEFENDER_FQDN\nDNS.3=twistlock-console\nIP.1=$CLUSTER_IP\nIP.2=127.0.0.1\n" > v3.ext

2. Create a defender server key

$ openssl genrsa -out defender-server-key.pem 4096

3. Generate a defender server CSR 

$ openssl req -new -sha256 -key defender-server-key.pem -subj "/OU=TwistlockCustomCert/CN=ConsoleWebsocketListener" -out defender-server-cert.csr

4. Create a defender server certificate signed by the root CA certificate

$ openssl x509 -req -in defender-server-cert.csr -CA rootCA.pem -CAkey rootCA-key.pem -CAcreateserial -out defender-server-cert.pem -days 3650 -sha256 -extfile v3.ext -passin file:mypass.enc


Step 4: Generate a defender-client certificate for the defender

1. Create the defender client key

$ openssl genrsa -out defender-client-key.pem 4096

2. Generate defender client CSR

$ openssl req -new -sha256 -key defender-client-key.pem -subj "/OU=TwistlockCustomCert/CN=DefenderWebsocketListener" -out defender-client-cert.csr

3. Create the defender client certificate

$ openssl x509 -req -in defender-client-cert.csr -CA rootCA.pem -CAkey rootCA-key.pem -CAcreateserial -out defender-client-cert.pem -days 3650 -sha256 -passin file:mypass.enc

 

Step 5: Generate a proxy certificate for the ingress (Optional)

1. Create the proxy key

$ openssl genrsa -out proxy-key.pem 4096

2. Generate the proxy CSR

$ openssl req -new -sha256 -key defender-client-key.pem -subj "/OU=TwistlockCustomCert/CN=ProxyWebsocketListener" -out proxy-cert.csr

3. Create the proxy certificate

$ openssl x509 -req -in proxy-cert.csr -CA rootCA.pem -CAkey rootCA-key.pem -CAcreateserial -out proxy-cert.pem -days 3650 -sha256 -extfile <(printf "subjectAltName=DNS:$DEFENDER_FQDN") -passin file:mypass.enc

 

This step is optional because you can substitute this certificate and key for the others you’ve generated, but is just as a best practice.

 

Step 6: Validate the certificates content

$ openssl x509 -text -noout -in defender-server-cert.pem
$ openssl x509 -text -noout -in defender-client-cert.pem
$ openssl x509 -text -noout -in proxy-cert.pem

 

You can check that the SAN, CN and OU values are correct.

 

Step 7: Secrets management for console, defender and proxy

1. Create custom certificates secret for the console

$ kubectl create secret generic console-custom-cert -n twistlock --from-file=defender-ca.pem=rootCA.pem --from-file=defender-server-cert.pem=defender-server-cert.pem --from-file=defender-server-key.pem=defender-server-key.pem

2. Add the following volume and volume mount to the file twistlock_console.yaml and apply changes:

 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: twistlock-console
  namespace: twistlock
  ...
spec:
  ...
  template:
    ...
    spec:
...
      containers:
      - name: twistlock-console
...
        volumeMounts:
        - name: custom-certificates
          mountPath: "/var/lib/twistlock/custom-certificates"
...
      volumes:
      - name: custom-certificates
        secret:
          secretName: console-custom-cert
          defaultMode: 256
...

Figure 8: Mounting custom certificates on the Console deployment_palo-alto-networks

 

$ kubectl apply -f twistlock_console.yaml

 

3. Create the custom certificates for the defender

$ kubectl create secret generic defender-custom-cert -n twistlock --from-file=defender-ca.pem=rootCA.pem --from-file=defender-client-cert.pem=defender-client-cert.pem --from-file=defender-client-key.pem=defender-client-key.pem

 

4. Add the following volume and volume mount to the file daemonset.yaml you’ve downloaded in step G:

  

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: twistlock-defender-ds
  namespace: twistlock
  ...
spec:
  ...
  template:
    ...
    spec:
...
      containers:
      - name: twistlock-defender
...
        volumeMounts:
        - name: custom-certificates
          mountPath: "/var/lib/twistlock/custom-certificates"
...
      volumes:
      - name: custom-certificates
        secret:
          secretName: defender-custom-cert
          defaultMode: 256
...

Figure 9: Mounting custom certificates on the Defender DaemonSet_palo-alto-networks

 

Generate secrets for Ingress 

$ kubectl create secret generic ca-secret -n twistlock --from-file=ca.crt=rootCA.pem
$ kubectl create secret generic proxy-ca-secret -n twistlock --from-file=ca.crt=rootCA.pem --from-file=tls.crt=proxy-cert.pem --from-file=tls.key=proxy-key.pem
$ kubectl create secret tls proxy-tls -n twistlock --cert proxy-cert.pem --key proxy-key.pem

You could replace the values proxy-cert.pem and proxy-key.pem from in secret proxy-ca-secret for defender-client-cert.pem and defender-client-key.pem respectively.

You could also replace the values proxy-cert.pem and proxy-key.pem from in secret proxy-tls for defender-server-cert.pem and defender-server-key.pem respectively.

 

Step 8: Create the Console Defender Ingress

Use the same process found in Part 2 - Step 4 - Console Certificates Methods .

 

Step 9: Deploy the Defender

Only apply the modified daemonset.yaml file on different kubernetes context

$ kubectl apply -f daemonset.yaml

 

Part 3: Troubleshooting

 

After you’ve performed the preceding steps, go to Manage > Defenders > Deployed Defenders. It should look like this:

 

RPrasadi_5-1697565996482.png

Figure 10: Verifying Defenders’ connectivity_palo-alto-networks

 

If this is not the case, follow up the next steps.

 

Step 1: Verify the DNS resolution for DEFENDER_FQDN

For this step and the following, it’s recommended to spin up a test deployment or pod inside the cluster and namespace where the defender is being deployed. In this case, will be created a ubuntu deployment inside the twistlock namespace.

$ kubectl create deploy --image ubuntu:latest -n twistlock ubuntu -- sleep "Infinity"

 

After being deployed and the container is running, access the container and install the package dnsutils:

$ kubectl exec -i -t deploy/ubuntu -n twistlock -- /bin/bash
$ apt update && apt install -y dnsutils

 

Now verify that the DNS resolution works for DEFENDER_FQDN:

$ nslookup defender.example.com

 

Correct answer:

Server:         x.x.x.x

Address:        x.x.x.x#53


Non-authoritative answer:

Name:   defender.example.com

Address: PUBLIC_IP


Incorrect answers:

# 1

Server:         x.x.x.x

Address:        x.x.x.x#53


server can't find defender.example.com: NXDOMAIN


# 2

Server:         x.x.x.x

Address:        x.x.x.x#53


Non-authoritative answer:

Name:   defender.example.com

Address: WRONG_PUBLIC_IP

Figure 11: Verifying DNS resolution_palo-alto-networks

 

If you receive the incorrect answer #1, check your cluster DNS settings.  

If you receive the incorrect answer #2, check your Domain Name records or wait until the DNS Record set has been renewed globally. 

If you receive a correct answer, proceed to the next step.

 

Step 2: Verify the Defender - Console connectivity

Inside the test pod, install the openssl command and test connectivity:

$ apt install -y openssl
$ openssl s_client --connect defender.jfmontufar.com:443

 

Correct answer:

CONNECTED(00000003)

depth=0 CORRECT SERVER CERTIFICATE PARAMETERS

verify error:num=18:self-signed certificate

verify return:1

depth=0 CORRECT SERVER CERTIFICATE PARAMETERS

verify return:1

...


Incorrect answers:

# 1

CONNECTED(00000003)

depth=0 WRONG SERVER CERTIFICATE PARAMETERS

verify error:num=18:self-signed certificate

verify return:1

depth=0 WRONG SERVER CERTIFICATE PARAMETERS

verify return:1

...


# 2

HANG UP

Figure 12: Test Defender - Console connectivity_palo-alto-networks

 

If you receive an incorrect answer #1, check the proxy-tls secret, or check the nginx ingress controller logs.

If you receive incorrect answer #2, check for any restriction on the firewall of the nodes, the security groups, access control lists or others that might restrict the connectivity.

If you receive a correct answer, continue with the next step.


Step 3: Test parameter proxy_ssl_name

Inside the annotation nginx.ingress.kubernetes.io/configuration-snippet of the defender_ingress.yaml you can try out any of the values that are set up as SAN like twistlock-console  or any other

  $ nginx.ingress.kubernetes.io/configuration-snippet: |
      proxy_ssl_name "twistlock-console";

 

If the iterative troubleshooting does not work, continue to step 4.

 

Step 4: Check NGINX ingress controller logs and Defender daemonset logs

Look for any misconfiguration on the nginx controller level:

$ kubectl logs deploy/ingress-nginx-controller -n ingress-nginx -f

And Defender daemonset level:

$ kubectl logs deploy/twistlock-defender-ds -n twistlock -f

 

Additional Information

  • This configuration works even if the certificates are expired.
  • For security purposes, it’s recommended to renew the certificates.

Conclusion

In this article, we showed you how to deploy the Prisma Cloud Compute console behind a NGINX ingress. We covered how to install the NGINX ingress controller, how to deploy and configure the console, how to create the ingresses for the console web UI and defender - console communication, how to deploy a DaemonSet defender and how to troubleshoot some common errors found in this process.  

We showed that for a successful communication between the defender and the console, it is required to manage mutual authentication between the defender - ingress and ingress - console communication. This mutual authentication can be performed using the certificates that the console already provides or any custom certificates. That’s it! 

 

References

[1] NGINX Ingress Controller

[2] Setup NGINX Ingress for EKS

[3] Create an ingress controller in AKS

[4] Ingress with NGINX controller on GKE

[5] NGINX Ingress Configuration

[6] Create Client-Server certificates

[7] Prisma Cloud configure custom certificates

[8] Cert-Manager

 

About the Authors

Juan Montufar and Sriram Choudary Nimmagadda are senior customer success engineers specializing in Prisma Cloud, Next-Generation Firewall, AWS, Azure, GCP, containers and Kubernetes. They use collaborative approaches to break down complex problems into solutions for global enterprise customers and leverage their multi-industry knowledge to inspire success.  

Rate this article:
  • 6158 Views
  • 0 comments
  • 1 Likes
Register or Sign-in
Labels
Article Dashboard
Version history
Last Updated:
‎10-22-2024 11:08 AM
Updated by: