Kubernetes
Kubernetes is an open-source container-orchestration system for automating application deployment, scaling, and management. It was originally designed by Google, and is now maintained by the Cloud Native Computing Foundation.
Summary
Tools
Container Environment
Information Gathering
RBAC Configuration
Listing Secrets
Access Any Resource or Verb
Pod Creation
Privilege to Use Pods/Exec
Privilege to Get/Patch Rolebindings
Impersonating a Privileged Account
Privileged Service Account Token
Interesting endpoints to reach
API addresses that you should know
References
Tools
Container Environment
Service Account
Each Kubernetes pod is assigned a service account for accessing the Kubernetes API. The service account, in addition to the current namespace and Kubernetes SSL certificate, are made available via a mounted read-only volume:
If the kubectl
utility is installed in the container, it will use this service account automatically and will make interacting with the cluster much easier. If not, the contents of the token
and namespace
files can be used to make HTTP API requests directly.
Environment Variables
The KUBERNETES_SERVICE_HOST
and KUBERNETES_SERVICE_PORT
environment variables are automatically provided to the container. They contain the IP address and port number of the Kubernetes master node. If kubectl
is installed, it will use these values automatically. If not, the values can be used to determine the correct IP address to send API requests to.
A simplified
{SVCNAME}_SERVICE_HOST
and{SVCNAME}_SERVICE_PORT
contain the IP address and default port number for the service.
For example, all of the following environment variables would be available if a redis-master
service were running with port 6379 exposed:
Simulating kubectl
API Requests
kubectl
API RequestsRun the desired command at the maximum verbosity level using
kubectl -v9 ...
The output will include HTTP API endpoint URL, the request body, and an example curl command.
Replace the endpoint URL's hostname and port with the
KUBERNETES_SERVICE_HOST
andKUBERNETES_SERVICE_PORT
values from the container's environment variables.Replace the masked "Authorization: Bearer" token value with the contents of
/var/run/secrets/kubernetes.io/serviceaccount/token
from the container.If the request had a body, ensure the "Content-Type: application/json" header is included and send the request body using the customary method (for curl, use the
--data
flag).
For example, this output was used to create the Service Account Permissions request:
Information Gathering
Service Account Permissions
The default service account may have been granted additional permissions that make cluster compromise or lateral movement easier. The following can be used to determine the service account's permissions:
Secrets, ConfigMaps, and Volumes
Kubernetes provides Secrets and ConfigMaps as a way to load configuration into containers at runtime. While they may not lead directly to whole cluster compromise, the information they contain can lead to individual service compromise or enable lateral movement within a cluster.
From a container perspective, Kubernetes Secrets and ConfigMaps are identical. Both can be loaded into environment variables or mounted as volumes. It's not possible to determine if an environment variable was loaded from a Secret/ConfigMap, so each environment variable will need to be manually inspected. When mounted as a volume, Secrets/ConfigMaps are always mounted as read-only tmpfs filesystems. You can quickly find these with grep -F "tmpfs ro" /etc/mtab
.
True Kubernetes Volumes are typically used as shared storage or for persistent storage across restarts. These are typically mounted as ext4 filesystems and can be identified with grep -wF "ext4" /etc/mtab
.
Privileged Containers
While there is no official way to check for privileged mode from within a container, checking if /dev/kmsg
exists will usually suffice.
RBAC Configuration
Listing Secrets
An attacker that gains access to list secrets in the cluster can use the following curl commands to get all secrets in "kube-system" namespace.
Access Any Resource or Verb
Pod Creation
Check your right with kubectl get role system:controller:bootstrap-signer -n kube-system -o yaml
. Then create a malicious pod.yaml file.
Then kubectl apply -f malicious-pod.yaml
Privilege to Use Pods/Exec
Privilege to Get/Patch Rolebindings
The purpose of this JSON file is to bind the admin "CluserRole" to the compromised service account. Create a malicious RoleBinging.json file.
Impersonating a Privileged Account
Privileged Service Account Token
Interesting endpoints to reach
API addresses that you should know
(External network visibility)
cAdvisor
Insecure API server
Secure API Server
etcd API
Kubelet API
kubelet (Read only)
References
Last updated