Runtime Security: Auto-deploy DaemonSet Defender using Kubernetes Service Account   

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
L2 Linker
No ratings

By Juan Montufar, Senior Customer Success Engineer

and

Omoniyi Jabaru, Senior Customer Success Engineer



Introduction 

 

Definition

The Prisma Cloud Runtime Security DaemonSet auto-deploy feature uses a kubeconfig file generated from a kubernetes service account with limited permissions. 

 

Purpose
If you aim to streamline the deployment of Defender DaemonSets to a cluster or lack direct kubectl access to your cluster, you can conveniently deploy Defender DaemonSets directly from the Console UI.

 

The Auto-Defend feature also allows you to upgrade with ease any Defender that you have deployed before, so you could easily perform the upgrade process from the Console UI or automate it by making API calls to the appropriate console endpoints. 


Kubernetes Service Account

 

Objective

Discover how to generate kubeconfig credentials, upload them to the console, and leverage them to automatically deploy Defender DaemonSets to a Kubernetes cluster, like Amazon EKS(Elastic Kubernetes Service), GKE (Google Kubernetes Service) or AKS (Azure Kubernetes Service), from within the console.

 

Environment

  • Prisma Cloud Compute Enterprise Edition

  • Prisma Cloud Compute Self-Hosted

  • Kubernetes Clusters (EKS, AKS, GKE or other Kubernetes cluster)

 

Requirements

This document primarily uses kubectl and assumes you are using bash or Z shell and have access to permissions that can create and/or update these resources in your Kubernetes cluster:

  • Kubernetes Service Account(s)

  • Kubernetes Roles and RoleBindings

  • Kubernetes ClusterRoles and ClusterRoleBindings

If you prefer working with API calls, please also make sure to have the JSON processor jq installed.

 

Part 1: Creating Service Account

 

Step 1: Create the service account manifest

Create a twistlock-service-account.yaml file with the following content:

 

apiVersion: v1

kind: Namespace

metadata:

  name: twistlock

---

apiVersion: v1

kind: ServiceAccount

metadata:

  name: twistlock-service-account

  namespace: twistlock

---

apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRole

metadata:

  name: twistlock-clusterrole

rules:

- apiGroups: 

  - "rbac.authorization.k8s.io"

  resources: 

  - "clusterroles"

  - "clusterrolebindings"

  verbs: 

  - "create"

  - "delete"

  - "list"

- apiGroups: 

  - ""

  resources:

  - "namespaces"

  verbs: 

  - "create"

  - "get"

- apiGroups:

  - "apps"

  - "rbac.authorization.k8s.io"

  resources: 

  - "daemonsets"

  - "roles"

  - "rolebindings"

  verbs:

  - "list"

- apiGroups:

  - ""

  - "apps"

  resources:

  - "deployments"

  - "replicasets"

  - "pods"

  verbs:

  - "get"

---

apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRoleBinding

metadata:

  name: twistlock-clusterrolebinding

subjects:

- kind: ServiceAccount

  name: twistlock-service-account

  namespace: twistlock

roleRef:

  kind: ClusterRole

  name: twistlock-clusterrole

  apiGroup: rbac.authorization.k8s.io

---

apiVersion: rbac.authorization.k8s.io/v1

kind: Role

metadata:

  name: twistlock-role

  namespace: twistlock

rules: 

- apiGroups: 

  - ""

  - "apps"

  - "rbac.authorization.k8s.io"

  resources: 

  - "daemonsets"

  - "secrets"

  - "serviceaccounts"

  - "services"

  verbs: 

  - "create"

  - "delete"

---

apiVersion: rbac.authorization.k8s.io/v1

kind: RoleBinding

metadata:

  name: twistlock-rolebinding

  namespace: twistlock

subjects:

- kind: ServiceAccount

  name: twistlock-service-account

  namespace: twistlock

roleRef:

  kind: Role

  name: twistlock-role

  apiGroup: rbac.authorization.k8s.io

Figure 1: twistlock service account manifest_palo-alto-networks

 

This file creates the following resources:

 

  • Namespace: creates the twistlock namespace.
  • Service account: creates a service account name twistlock-service-account that would be used to grant permissions so the console can deploy the defender.
  • ClusterRole: a role to grant permissions on Cluster level.
  • ClusterRoleBinding: assign permissions from the ClusterRole created to the twistlock-service-account.
  • Role: a role to grant permissions on namespace level.
  • RoleBinding: assign permissions from the Role created to the twistlock-service-account.

 

To enable Istio monitoring, the  twistlock-service-account.yaml needs to have the following content:

 

apiVersion: v1

kind: Namespace

metadata:

  name: twistlock

---

apiVersion: v1

kind: ServiceAccount

metadata:

  name: twistlock-service-account

  namespace: twistlock

---

apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRole

metadata:

  name: twistlock-clusterrole

rules:

- apiGroups: 

  - "rbac.authorization.k8s.io"

  resources: 

  - "clusterroles"

  - "clusterrolebindings"

  verbs: 

  - "create"

  - "delete"

  - "list"

- apiGroups: 

  - ""

  resources:

  - "namespaces"

  verbs: 

  - "create"

  - "get"

- apiGroups:

  - ""

  - "apps"

  - "rbac.authorization.k8s.io"

  - "networking.istio.io"

  - "security.istio.io"

  resources: 

  - "daemonsets"

  - "roles"

  - "rolebindings"

  - "endpoints"

  - "pods"

  - "services"

  - "destinationrules"

  - "gateways"

  - "virtualservices"

  - "authorizationpolicies"

  - "peerauthentications"

  verbs:

  - "list"

- apiGroups:

  - ""

  - "apps"

  resources:

  - "deployments"

  - "replicasets"

  - "pods"

  - "pods/proxy"

  verbs:

  - "get"

---

apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRoleBinding

metadata:

  name: twistlock-clusterrolebinding

subjects:

- kind: ServiceAccount

  name: twistlock-service-account

  namespace: twistlock

roleRef:

  kind: ClusterRole

  name: twistlock-clusterrole

  apiGroup: rbac.authorization.k8s.io

---

apiVersion: rbac.authorization.k8s.io/v1

kind: Role

metadata:

  name: twistlock-role

  namespace: twistlock

rules: 

- apiGroups: 

  - ""

  - "apps"

  - "rbac.authorization.k8s.io"

  resources: 

  - "daemonsets"

  - "secrets"

  - "serviceaccounts"

  - "services"

  verbs: 

  - "create"

  - "delete"

---

apiVersion: rbac.authorization.k8s.io/v1

kind: RoleBinding

metadata:

  name: twistlock-rolebinding

  namespace: twistlock

subjects:

- kind: ServiceAccount

  name: twistlock-service-account

  namespace: twistlock

roleRef:

  kind: Role

  name: twistlock-role

  apiGroup: rbac.authorization.k8s.io

Figure 2: twistlock service account manifest with Istio monitoring_palo-alto-networks

 

Step 2: Apply service account manifest

Apply the twistlock-service-account.yaml on to the cluster where the defender is going to be deployed by running the following command:

$ kubectl apply -f twistlock-service-account.yaml

 

Part 2: Generating Token 

 

Temporary Token Method

 

Step 1: Generate temporary token for service account

Run the following command to generate an environment variable named TOKEN :

$ export TOKEN=$(kubectl create token twistlock-service-account -n twistlock --duration 86400s)

 

The token generated can have a one day maximum expiration period.

 

Non-Expirable Token Method

 

Step 1: Create service account secret manifest

Create the file twistlock-sa-secret.yaml with the following content: 

 

apiVersion: v1

kind: Secret

metadata:

  name: twistlock-sa-secret

  namespace: twistlock

  annotations:

    kubernetes.io/service-account.name: twistlock-service-account

type: kubernetes.io/service-account-token

Figure 3: twistlock service account secret manifest_palo-alto-networks

 

Step 2: Apply service account secret manifest

Apply the twistlock-sa-secret.yaml by running the following command:

$ kubectl apply -f twistlock-sa-secret.yaml

 

Step 3: Generate non-expirable token for service account

Run the following command to generate an environment variable named TOKEN:

$ export TOKEN=$(kubectl get secret twistlock-sa-secret -o  jsonpath='{$.data.token}' -n twistlock | base64 -d)


Part 3: Generating kubeconfig file for Service Account

 

Step 1: Create installation script

Create a file named generate-kubeconfig.sh with the following content:

 

#!/usr/bin/bash

# Create a full copy

kubectl config view --raw > ${KUBECONFIG_FILE}.full.tmp

# Switch working context to correct context

kubectl --kubeconfig ${KUBECONFIG_FILE}.full.tmp config use-context ${CONTEXT}

# Minify

kubectl --kubeconfig ${KUBECONFIG_FILE}.full.tmp \

  config view --flatten --minify > ${KUBECONFIG_FILE}.tmp

# Rename context

kubectl config --kubeconfig ${KUBECONFIG_FILE}.tmp \

  rename-context ${CONTEXT} ${NEW_CONTEXT}

# Create token user

kubectl config --kubeconfig ${KUBECONFIG_FILE}.tmp \

  set-credentials ${CONTEXT}-${NAMESPACE}-token-user \

  --token ${TOKEN}

# Set context to use token user

kubectl config --kubeconfig ${KUBECONFIG_FILE}.tmp \

  set-context ${NEW_CONTEXT} --user ${CONTEXT}-${NAMESPACE}-token-user

# Set context to correct namespace

kubectl config --kubeconfig ${KUBECONFIG_FILE}.tmp \

  set-context ${NEW_CONTEXT} --namespace ${NAMESPACE}

# Flatten/minify kubeconfig

kubectl config --kubeconfig ${KUBECONFIG_FILE}.tmp \

  view --flatten --minify > ${KUBECONFIG_FILE}

# Remove tmp

rm ${KUBECONFIG_FILE}.full.tmp

rm ${KUBECONFIG_FILE}.tmp

Figure 4: generate-kubeconfig.sh file content_palo-alto-networks

 

We are going to inject all the variables as environment variables with the command envsubst. 

 

Step 2: Setup the environment variables

Run the following commands to setup the required environment variables:

$ export SERVICE_ACCOUNT_NAME=twistlock-service-account

$ export CONTEXT=$(kubectl config current-context)

$ export NAMESPACE=twistlock

$ export NEW_CONTEXT=twistlock-context

$ export KUBECONFIG_FILE="kubeconfig-sa"

 

The variable TOKEN was already created in Part 2.

 

Step 3: Generate the kubeconfig file 

Run the following command:

$ envsubst < generate-kubeconfig.sh | sh

 

This will generate a file in your local directory with the name of the variable KUBECONFIG_FILE.


Part 4: Auto-Deploy Defender with kubeconfig

 

You can perform the auto-deployment of the DaemonSet defender via the Prisma Cloud Compute console UI, or via its API for automation purposes. You can choose which method suits you better.

 

Console UI Method

 

Step 1: Import kubeconfig file into the Console UI

​​​On the Prisma Cloud Compute console, Navigate to Manage > Authentication > Credentials store and click on the ‘+ Add credential’ button:

 

RPrasadi_0-1698857252513.png

Figure 5: Navigate to add credentials_palo-alto-networks


At the Create new credential screen, add the following parameters:

 

  1. Name: Any of your choice
  2. Description: Any of your choice
  3. Type: kubeconfig
  4. Kubeconfig: the content of the kubeconfig file generated in Part 3 - Step 3.

 

RPrasadi_1-1698857252439.png

Figure 6: adding credentials on Prisma Cloud Compute console_palo-alto-networks

 

Click on Save.

 

Step 2: Auto-Deploy DaemonSet Defender on the Console UI

Navigate to Manage > Defenders > Auto-defend > DaemonSet. You will notice that the credential you have created will now be shown on the screen.

 

RPrasadi_2-1698857251468.png

Figure 7: Navigate to DaemonSet Auto-defend_palo-alto-networks

 

Click on the Deploy symbol under the Actions column and fill up the settings required for your kubernetes cluster. Once done click on Deploy.

 

RPrasadi_3-1698857252367.png

Figure 8: Auto-deploy Defender settings_palo-alto-networks

 

Now you should be able to see the new defenders deployed under Manage > Defenders > Deployed Defenders.

 

RPrasadi_4-1698857251878.png

Figure 9: New DaemonSet Defenders deployed_palo-alto-networks


Console API Method

 

Step 1: Import kubeconfig file using Console API

Get the console token running the following commands:

$ CONSOLE_URL="https://console.example.com:8083"

$ USERNAME="sample_user"

$ PASSWORD="sample_password"

$ CONSOLE_TOKEN=$(curl -k ${CONSOLE_URL}/api/v1/authenticate -X POST -H "Content-Type: application/json" -d '{"username":"'"$USERNAME"'", "password":"'"$PASSWORD"'"}'  | jq -r '.token')

 

Then upload the kubeconfig to Prisma Cloud Compute, by running the following commands:

$ KUBECONFIG_NAME="k8s-cluster"

$ KUBECONFIG=$(sed -z 's/\n/\\n/g; s/..$//' $KUBECONFIG_FILE)

$ curl -k ${CONSOLE_URL}/api/v1/credentials -H 'Content-Type: application/json' -H "Authorization: Bearer $CONSOLE_TOKEN" -d '{

  "secret":{

      "plain":"'"$KUBECONFIG"'"

  },

  "description": "",

  "type": "kubeconfig",

  "_id": "'"$KUBECONFIG_NAME"'"

}'

 

NOTE: This uses the variable KUBECONFIG_FILE  from Part 3 - Step 2.

 

Step 2:  Auto-Deploy DaemonSet Defender using Console API

 

Run the following commands:

 

$ CONSOLE_ADDRESS=”console-address.example.com”

$ curl -k ${CONSOLE_URL}/api/v1/deployment/daemonsets/deploy -H 'Content-Type: application/json' -H "Authorization: Bearer $CONSOLE_TOKEN" -d '{

  "consoleAddr":"'"$CONSOLE_ADDRESS"'",

  "namespace": "'"$NAMESPACE"'", 

  "containerRuntime": "containerd",

  "orchestration": "kubernetes",

  "credentialID": "'"$KUBECONFIG_NAME"'",

  "privileged": false, 

  "serviceAccounts": true, 

  "istio": false, 

  "collectPodLabels": true, 

  "proxy": null, 

  "taskName": null

}'

 

You can find more information about the body content here.

 

Part 5: Upgrade DeamonSet Defender

 

Step 1:  Renew Token (optional)

If you have used the temporary token method of Part 2, only rerun the command found in Step 1.

 

If you have used the non-expirable token method of Part 2, you don’t require to renew such token, but if you still want to by following best practices or any internal compliance, you can follow up the next commands:

 

$ kubectl delete secret twistlock-sa-secret -n twistlock

$ kubectl apply -f twistlock-sa-secret.yaml

$ export TOKEN=$(kubectl get secret twistlock-sa-secret -o jsonpath='{$.data.token}' -n twistlock | base64 -d)

 

Step 2:  Update kubeconfig file (optional)

If you have renewed the token, you can update the kubeconfig by running the next commands:

$ CONTEXT=$(kubectl config current-context)

$ NAMESPACE=twistlock

$ KUBECONFIG_FILE="kubeconfig-sa"

$ kubectl config --kubeconfig ${KUBECONFIG_FILE} \

  set-credentials ${CONTEXT}-${NAMESPACE}-token-user \

  --token ${TOKEN}

 

Step 3:  Upload updated kubeconfig file (optional)

If you have renewed the token, you need to upload the kubeconfig file again with the new token. For that,you can follow the Part 4 - Console UI Method - Step 1 or Part 4 - Console API Method - Step 1

 

Step 4:  Re-deploy DaemonSet Defender

For such a process you can follow the Part 4 - Console UI Method - Step 2 or Part 4 - Console API Method - Step 2

 

Additional Information 

 

kubeconfig is a YAML file that contains either a username and password combination or a secure token that when read programmatically removes the need for the Kubernetes client to ask for interactive authentication. kubeconfig is the secure and standard method to enable access to your Kubernetes clusters.

 

Conclusion

 

In this article, we showed how you can simplify the Prisma Cloud Runtime Security defender upgrade process in any kubernetes environment while using the bare minimum set of permissions. We achieved that by making use of the auto-deploy feature for DaemonSet Defenders and a kubernetes service account.

 

References

 

[1] Spinnaker

 

[2] Configure Service Account

 

[3] Provision Elastic Kubernetes Clusters (EKS Clusters)

 

[4] Provision Google Kubernetes Clusters (GKE Clusters)

 

[5] Provision Azure Kubernetes Clusters (AKS Clusters)

 

[6] Prisma Credential Store

 

[7] Generate DaemonSet yaml

 

About the Authors

 

Juan Montufar and Omoniyi Jabaru are senior customer success engineers specializing in Prisma Cloud, Next-Generation Firewall, AWS, Azure, GCP, containers and Kubernetes. They use collaborative approaches to break down complex problems into solutions for global enterprise customers and leverage their multi-industry knowledge to inspire success.

 

Rate this article:
(1)
Comments
Community Team Member

Thanks for sharing team!

L1 Bithead

Error fetching Daemonset: Get "https://EKS-Control-Plane-URL/apis/apps/v1/daemonsets?fieldselector=metadata.name%3Dtwistlock-defend...": dial tcp IPaddress:443:i/o timeout.

 

Pls advise.

L1 Bithead

Any comment/update on the below error?

 

Error fetching Daemonset: Get "https://EKS-Control-Plane-URL/apis/apps/v1/daemonsets?fieldselector=metadata.name%3Dtwistlock-defend...": dial tcp IPaddress:443:i/o timeout.

L1 Bithead

Does no one reviews this article as there is no response at all?

L2 Linker

@laurenlin 
The error is generic, and we need more details to help resolve the issue. Do you have access to Prisma Cloud Support?. 

L1 Bithead

@JOmoniyi pls advise what more details are required? I've followed the article and able to upload the kubeconfig file as an authentication but when I go to the Defenders:Auto-defend tab under Manage DaemonSets it loads and then throws the error as highlighted below.

 

Error fetching Daemonset: Get "https://EKS-Control-Plane-URL/apis/apps/v1/daemonsets?fieldselector=metadata.name%3Dtwistlock-defend...": dial tcp IPaddress:443:i/o timeout.

 

PFA the screenshot.

 

K8S.PNG

 

L1 Bithead

Seems like a connectivity issue. Console is not able to reach the cluster. Is there any firewall rule?

  • 5795 Views
  • 7 comments
  • 3 Likes
Register or Sign-in
Labels
Article Dashboard
Version history
Last Updated:
‎11-03-2023 04:41 PM
Updated by: