GitBook: [master] 454 pages modified

This commit is contained in:
CPol 2021-04-28 23:33:12 +00:00 committed by gitbook-bot
parent 89dbdd726a
commit 2c77993d8b
No known key found for this signature in database
GPG Key ID: 07D2180C7B12D0FF
5 changed files with 178 additions and 268 deletions

View File

@ -488,19 +488,33 @@ $ mount /dev/sda1 /mnt-testmount: /mnt: permission denied. ---> Failed! but if n
$ debugfs /dev/sda1
```
## Seccomp in Docker
## Containers Security Improvements
### Seccomp in Docker
This is not a technique to breakout from a Docker container but a security feature that Docker uses and you should know about as it might prevent you from breaking out from docker:
{% page-ref page="seccomp.md" %}
## AppArmor in Docker
### AppArmor in Docker
This is not a technique to breakout from a Docker container but a security feature that Docker uses and you should know about as it might prevent you from breaking out from docker:
{% page-ref page="apparmor.md" %}
## Use containers securely
### gVisor
**gVisor** is an application kernel, written in Go, that implements a substantial portion of the Linux system surface. It includes an [Open Container Initiative \(OCI\)](https://www.opencontainers.org/) runtime called `runsc` that provides an **isolation boundary between the application and the host kernel**. The `runsc` runtime integrates with Docker and Kubernetes, making it simple to run sandboxed containers.
{% embed url="https://github.com/google/gvisor" %}
## Kata Containers
**Kata Containers** is an open source community working to build a secure container runtime with lightweight virtual machines that feel and perform like containers, but provide **stronger workload isolation using hardware virtualization** technology as a second layer of defense.
{% embed url="https://katacontainers.io/" %}
### Use containers securely
Docker restricts and limits containers by default. Loosening these restrictions may create security issues, even without the full power of the `--privileged` flag. It is important to acknowledge the impact of each additional permission, and limit permissions overall to the minimum necessary.

View File

@ -202,7 +202,7 @@ nmap -sV --script "docker-*" -p <PORT> <IP>
## Compromising
In the following page you can find a way to **scape from a docker container**:
In the following page you can find ways to **escape from a docker container**:
{% page-ref page="../linux-unix/privilege-escalation/docker-breakout.md" %}
@ -219,6 +219,27 @@ cat /mnt/etc/shadow
If you are inside a host that is using docker, you may [**read this information to try to elevate privileges**](../linux-unix/privilege-escalation/#writable-docker-socket).
## Discovering secrets in running Docker containers
```bash
docker ps [| grep <kubernetes_service_name>]
docker inspect <docker_id>
```
Check **env** \(environment variable section\) for secrets and you may find:
* Passwords.
* Ips.
* Ports.
* Paths.
* Others… .
If you want to extract a file:
```bash
docker cp <docket_id>:/etc/<secret_01> <secret_01>
```
## Securing your Docker
### Securing Docker installation and usage

View File

@ -1,6 +1,6 @@
# Pentesting Kubernetes
**The main author of this page is** [**Jorge**](https://www.linkedin.com/in/jorge-belmonte-a924b616b/) **\(read his original post** [**here**](https://sickrov.github.io/)**\)**
**The original author of this page is** [**Jorge**](https://www.linkedin.com/in/jorge-belmonte-a924b616b/) **\(read his original post** [**here**](https://sickrov.github.io/)**\)**
## Architecture & Basics
@ -355,17 +355,19 @@ Helm is also a template engine that allows to generate config files with variabl
{% page-ref page="pentesting-kubernetes-from-the-outside.md" %}
## VULNERABILITIES and some fixes
## Vulnerabilities and misconfigurations
### Enumeration inside a Pod
If you manage to **compromise a Pod** read the following page to learn how to enumerate and try to **escalate privileges/escape**:
{% page-ref page="enumeration-from-a-pod.md" %}
### Vulnerabilities - kubernetes secrets
### Kubernetes secrets
A Secret is an object that contains sensitive data such as a password, a token or a key. Such information might otherwise be put in a Pod specification or in an image. Users can create Secrets and the system also creates Secrets. The name of a Secret object must be a valid **DNS subdomain name**.
A **Secret** is an object that **contains sensitive data** such as a password, a token or a key. Such information might otherwise be put in a Pod specification or in an image. Users can create Secrets and the system also creates Secrets. The name of a Secret object must be a valid **DNS subdomain name**. Read here [the official documentation](https://kubernetes.io/docs/concepts/configuration/secret/).
Secrets can be things like:
Secrets might be things like:
* API, SSH Keys.
* OAuth tokens.
@ -373,11 +375,11 @@ Secrets can be things like:
* Information or comments.
* Database connection code, strings… .
Secret types:
There are different types of secrets in Kubernetes
| Builtin Type | Usage |
| :--- | :--- |
| Opaque | arbitrary user-defined data |
| **Opaque** | **arbitrary user-defined data \(Default\)** |
| kubernetes.io/service-account-token | service account token |
| kubernetes.io/dockercfg | serialized ~/.dockercfg file |
| kubernetes.io/dockerconfigjson | serialized ~/.docker/config.json file |
@ -386,150 +388,71 @@ Secret types:
| kubernetes.io/tls | data for a TLS client or server |
| bootstrap.kubernetes.io/token | bootstrap token data |
**How secrets works:**
{% hint style="info" %}
**The Opaque type is the default one, the typical key-value pair defined by users.**
{% endhint %}
[https://kubernetes.io/docs/concepts/configuration/secret/\#using-secrets-as-files-from-a-pod](https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets-as-files-from-a-pod)
**How secrets works:**
![](https://sickrov.github.io/media/Screenshot-164.jpg)
Create a secret, commands:
```bash
kubectl create secret generic secret_01 --from-literal user=<user>
kubectl create secret generic secret_01 --from-literal password=<password>
kubectl run pod --image=nginx -oyaml --dry-run=client
kubectl run pod --image=nginx -oyaml --dry-run=client > <podName.yaml>
```
This is the generated file:
The following configuration file defines a **secret** called `mysecret` with 2 key-value pairs `username: YWRtaW4=` and `password: MWYyZDFlMmU2N2Rm`. It also defines a **pod** called `secretpod` that will have the `username` and `password` defined in `mysecret` exposed in the **environment variables** `SECRET_USERNAME` __and __`SECRET_PASSWOR`. It will also **mount** the `username` secret inside `mysecret` in the path `/etc/foo/my-group/my-username` with `0640` permissions.
{% code title="secretpod.yaml" %}
```yaml
apiVersion: v1
kind: Secret
metadata:
name: mysecret
type: Opaque
data:
username: YWRtaW4=
password: MWYyZDFlMmU2N2Rm
---
apiVersion: v1
kind: Pod
metadata:
name: mypod
name: secretpod
spec:
containers:
- name: mypod
image: redis
volumeMounts:
- name: <secret_01>
mountPath: "/etc/<secret_01>"
readOnly: true
volumes:
- name: <secret_01>
secret:
secretName: <secret_01>
items:
- key: username
path: my-group/my-username
```
### Using Secrets as environment variables
If you want to use a secret in an environment variable to allow the rest of the pods to reference the same secret, you could use:
In the you could add the uncomment lines:
```yaml
#apiVersion: v1
#kind: Pod
#metadata:
# name: secret-env-pod
#spec:
# containers:
# - name: mycontainer
# image: redis
- name: secretpod
image: nginx
env:
- name: SECRET_USERNAME
valueFrom:
secretKeyRef:
name: mysecret
key: username
# - name: SECRET_PASSWORD
# valueFrom:
# secretKeyRef:
# name: mysecret
# key: password
# restartPolicy: Never
```
The result is:
```yaml
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: mypod
image: redis
env:
- name: PASSWORD
- name: SECRET_PASSWORD
valueFrom:
secretKeyRef:
name: <secret_02>
key: <password>
name: mysecret
key: password
volumeMounts:
- name: <secret_01>
mountPath: "/etc/<secret_01>"
readOnly: true
- name: foo
mountPath: "/etc/foo"
restartPolicy: Never
volumes:
- name: <secret_01>
- name: foo
secret:
secretName: <secret_01>
secretName: mysecret
items:
- key: username
path: my-group/my-username
mode: 0640
```
Save and:
{% endcode %}
```bash
kubectl -f <podName.yaml> delete --force
kubectl -f <podName.yaml> create
kubectl apply -f <secretpod.yaml>
kubectl get pods #Wait until the pod secretpod is running
kubectl exec -it secretpod -- bash
env | grep SECRET && cat /etc/foo/my-group/my-username && echo
```
or:
### Secrets in etcd <a id="discover-secrets-in-etcd"></a>
```bash
kubectl -f <podName.yaml> replace --force
```
More info: [https://kubernetes.io/docs/concepts/configuration/secret/\#using-secrets-as-environment-variables](https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets-as-environment-variables)
### Discover secrets in docker:
To get the id of the container.
```bash
docker ps | grep <service>
```
Inspect:
```bash
docker inspect <docker_id>
```
Check env \(environment variable section\) for secrets and you will see:
* Passwords.
* Ips.
* Ports.
* Paths.
* Others… .
If you want to copy:
```bash
docker cp <docket_id>:/etc/<secret_01> <secret_01>
```
### Discover secrets in etcd: <a id="discover-secrets-in-etcd"></a>
Remember that etcd is a consistent and highly-available key-value store used as Kubernetes backing store for all cluster data. Lets access to the secret in etcd:
**etcd** is a consistent and highly-available **key-value store** used as Kubernetes backing store for all cluster data. Lets access to the secrets stored in etcd:
```bash
cat /etc/kubernetes/manifests/kube-apiserver.yaml | grep etcd
@ -551,32 +474,11 @@ Once you achieve establish communication you would be able to get the secrets:
ETCDCTL_API=3 etcdctl --cert /etc/kubernetes/pki/apiserver-etcd-client.crt --key /etc/kubernetes/pki/apiserver-etcd-client.key --cacert /etc/kubernetes/pki/etcd/etcd/ca.cert endpoint=[127.0.0.1:1234] get /registry/secrets/default/secret_02
```
### Adding encryption to the ETCD <a id="adding-encryption-to-the-etcd"></a>
#### Adding encryption to the ETCD
So, by default all the secrets are in plain text unless you apply an encryption layer: If the identity provider is empty with the default value = {} so the secrets are in plain text. [https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/)
**Encryption types**
\| Name \| Encryption \| Strength \| Speed \| Key Length \| Other Considerations \| \|-\|-\|-\|-\|-\|-\| \| identity \| None \| N/A \| N/A \| N/A \| Resources written as-is without encryption. When set as the first provider, the resource will be decrypted as new values are written. \| \| aescbc \| AES-CBC with PKCS\#7 padding \| Strongest \| Fast \| 32-byte \| The recommended choice for encryption at rest but may be slightly slower than secretbox. \| \| secretbox \| XSalsa20 and Poly1305 \| Strong \| Faster \| 32-byte \| A newer standard and may not be considered acceptable in environments that require high levels of review. \| \| aesgcm \| AES-GCM with random nonce \| Must be rotated every 200k writes \| Fastest \| 16, 24, or 32-byte \| Is not recommended for use except when an automated key rotation scheme is implemented. \| \| kms \| Uses envelope encryption scheme: Data is encrypted by data encryption keys \(DEKs\) using AES-CBC with PKCS\#7 padding, DEKs are encrypted by key encryption keys \(KEKs\) according to configuration in Key Management Service \(KMS\) \| Strongest \| Fast \| 32-bytes \| The recommended choice for using a third party tool for key management. Simplifies key rotation, with a new DEK generated for each encryption, and KEK rotation controlled by the user. \|
The secrets will be encrypted with the above algorithms and encoded by base64.
```bash
kubectl get secrets --all-namespaces -o json | kubectl replace -f -
```
### **How to encrypt the ETCD**
Create a directory in /etc/kubernetes ; in this case you will name it as etcd, so you have `/etc/kubernetes/etcd`
You create a yaml file with the configuration.
```text
vi <configFile.yaml>
```
You can copy the content of [https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/)
By default all the secrets are **stored in plain** text inside etcd unless you apply an encryption layer. The following example is based on [https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/)
{% code title="encryption.yaml" %}
```yaml
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
@ -587,19 +489,12 @@ resources:
- aescbc:
keys:
- name: key1
secret: <your pass in b64>
secret: cjjPMcWpTPKhAdieVtd+KhG4NN+N6e3NmBPMXJvbfrY= #Any random key
- identity: {}
```
{% endcode %}
Generate pass in b64 \(remember to use a pass character with lenght = 16 or = 24 or = 32\) :
```bash
echo -n <password> | base64
```
You can see how the encryption provider is not setting.
After that, you have to edit the file `/etc/kubernetes/manifest/kube-apiserver.yaml` and add the following lines into the sections: And add the following line: spec:
After that, you need to set the `--encryption-provider-config` flag on the `kube-apiserver` to point to the location of the created config file. You can modify `/etc/kubernetes/manifest/kube-apiserver.yaml` and add the following lines:
```yaml
containers:
@ -625,24 +520,37 @@ Scroll down in the volumeMounts to hostPath:
name: etcd
```
### Get information about the secrets.
#### Verifying that data is encrypted
```bash
kubectl get secret
kubectl get secret <secret_name> -oyaml
ETCDCTL_API=3 etcdctl get /registry/secrets/default/secret1 [...] | hexdump -C
kubectl create secret generic test-secret --from-literal='username=my-app' --from-literal='password=45tRf$we34rR'
Data is encrypted when written to etcd. After restarting your `kube-apiserver`, any newly created or updated secret should be encrypted when stored. To check, you can use the `etcdctl` command line program to retrieve the contents of your secret.
1. Create a new secret called `secret1` in the `default` namespace:
```text
kubectl create secret generic secret1 -n default --from-literal=mykey=mydata
```
2. Using the etcdctl commandline, read that secret out of etcd:
`ETCDCTL_API=3 etcdctl get /registry/secrets/default/secret1 [...] | hexdump -C`
where `[...]` must be the additional arguments for connecting to the etcd server.
3. Verify the stored secret is prefixed with `k8s:enc:aescbc:v1:` which indicates the `aescbc` provider has encrypted the resulting data.
4. Verify the secret is correctly decrypted when retrieved via the API:
```text
kubectl describe secret secret1 -n default
```
should match `mykey: bXlkYXRh`, mydata is encoded, check [decoding a secret](https://kubernetes.io/docs/concepts/configuration/secret#decoding-a-secret) to completely decode the secret.
**Since secrets are encrypted on write, performing an update on a secret will encrypt that content:**
```text
kubectl get secrets --all-namespaces -o json | kubectl replace -f -
```
With root access:
```bash
# kubectl get secret
kubectl get secret test-secret -oyaml
```
Do not forget to delete de secrets and re-create them again in order to apply the encryption layer.
#### Final tips:
* Try not to keep secrets in the FS, get them from other places.
@ -650,38 +558,6 @@ Do not forget to delete de secrets and re-create them again in order to apply th
* [https://kubernetes.io/docs/concepts/configuration/secret/\#risks](https://kubernetes.io/docs/concepts/configuration/secret/#risks)
* [https://docs.cyberark.com/Product-Doc/OnlineHelp/AAM-DAP/11.2/en/Content/Integrations/Kubernetes\_deployApplicationsConjur-k8s-Secrets.htm](https://docs.cyberark.com/Product-Doc/OnlineHelp/AAM-DAP/11.2/en/Content/Integrations/Kubernetes_deployApplicationsConjur-k8s-Secrets.htm)
### Vulnerabilities - Container runtime sandboxes <a id="vulnerabilities---container-runtime-sandboxes"></a>
How an attack with lateral movement and privesc could be done:
![](https://sickrov.github.io/media/Screenshot-161.jpg)
Getting inside the container:
```bash
kubectl get node
kubectl run pod --image=<image_name>
kubectl exec pod -it -- bash
```
Once inside the container:
```text
uname -r
```
When the attack achieves discovering the kernel version, he could run exploiting techniques to gather information or escalate into the OS.
For secure sandboxes:
* gVisor:
[https://github.com/google/gvisor](https://github.com/google/gvisor)
* Katakontainers:
[https://katacontainers.io/](https://katacontainers.io/)
### Vulnerabilities - OS <a id="vulnerabilities---os"></a>
Is mandatory to keep in mind to define privilege and access control for container / pod:
@ -775,60 +651,7 @@ Inside you add in
- --enable-admission-plugins=NodeRestriction,PodSecurityPolicy
```
### Vulnerabilities - mTLS <a id="vulnerabilities---mtls"></a>
Mutual authentication, two-way, pod to pod.
![](https://sickrov.github.io/media/Screenshot-165.jpg)
More info at: [https://kubernetes.io/docs/tasks/configure-pod-container/security-context/](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/)
#### Create a sidecar proxy app <a id="create-a-sidecar-proxy-app"></a>
Create your .yaml
```bash
kubectl run app --image=bash --comand -oyaml --dry-run=client > <appName.yaml> -- shj -c 'ping google.com'
```
Edit your .yaml and add the uncomment lines:
```yaml
#apiVersion: v1
#kind: Pod
#metadata:
# name: security-context-demo
#spec:
# securityContext:
# runAsUser: 1000
# runAsGroup: 3000
# fsGroup: 2000
# volumes:
# - name: sec-ctx-vol
# emptyDir: {}
# containers:
# - name: sec-ctx-demo
# image: busybox
command: [ "sh", "-c", "apt update && apt install iptables -y && iptables -L && sleep 1h" ]
securityContext:
capabilities:
add: ["NET_ADMIN"]
# volumeMounts:
# - name: sec-ctx-vol
# mountPath: /data/demo
# securityContext:
# allowPrivilegeEscalation: true
```
See the logs of the proxy:
```bash
kubectl logs app -C proxy
```
More info at: [https://kubernetes.io/docs/tasks/configure-pod-container/security-context/](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/)
## CLUSTER HARDENING - RBAC
## RBAC Hardening
Kubernetes has an **authorization module named Role-Based Access Control** \([**RBAC**](https://kubernetes.io/docs/reference/access-authn-authz/rbac/)\) that helps to set utilization permissions to the API server.
The RBAC table is constructed from “**Roles**” and “**ClusterRoles**.” The difference between them is just where the role will be applied a “**Role**” will grant access to only **one** **specific** **namespace**, while a “**ClusterRole**” can be used in **all namespaces** in the cluster. Moreover, ClusterRoles can also grant access to:
@ -948,13 +771,13 @@ This is enabled by default. RBAC functions:
When configuring roles and permissions it's highly important to always follow the principle of Least Privileges
{% endhint %}
## SERVICE ACCOUNTS HARDENING
## Service Accounts Hardening
**To learn about Service Accounts Hardenig read the page:**
{% page-ref page="enumeration-from-a-pod.md" %}
## KUBERNETES API HARDENING
## Kubernetes API Hardening
It's very important to **protect the access to the Kubernetes Api Server** as a malicious actor with enough privileges could be able to abuse it and damage in a lot of way the environment.
It's important to secure both the **access** \(**whitelist** origins to access the API Server and deny any otehr connection\) and the [**authentication**](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet-authentication-authorization/) \(following the principle of **least** **privilege**\). And definitely **never** **allow** **anonymous** **requests**.
@ -976,7 +799,7 @@ User or K8s ServiceAccount &gt; Authentication &gt; Authorization &gt;
* Avoid unauthorized access RBAC.
* ApiServer port with firewall and IP whitelisting.
## KUBERNETES CLUSTER HARDENING
## General Hardening
You should update your Kubernetes environment as frequently as necessary to have:

View File

@ -71,6 +71,12 @@ GET /apis/apps/v1/watch/deployments [DEPRECATED]
They open a streaming connection that returns you the full manifest of a Deployment whenever it changes \(or when a new one is created\).
{% hint style="danger" %}
The following `kubectl` commands indicates just how to list the objects. If you want to access the data you need to add the option `-o json` or `-o yaml`
{% endhint %}
### Get namespaces
{% tabs %}
@ -88,16 +94,13 @@ https://<Kubernetes_API_IP>:<port>/api/v1/namespaces/
{% endtab %}
{% endtabs %}
### Get/List secrets
### Get secrets
{% tabs %}
{% tab title="kubectl" %}
```text
./kubectl get secrets -o yaml
./kubectl get secrets -o yaml -n custnamespace
./kubectl list secrets -o yaml
./kubectl list secrets -o yaml -n custnamespace
```
{% endtab %}
@ -227,8 +230,59 @@ https://<Kubernetes_API_IP>:<port>/apis/extensions/v1beta1/namespaces/default/da
**If you are lucky enough you may be able to escape from it to the node:**
![](https://sickrov.github.io/media/Screenshot-161.jpg)
{% page-ref page="../../linux-unix/privilege-escalation/docker-breakout.md" %}
## Sniffing
By default there isn't any encryption in the communication between pods .Mutual authentication, two-way, pod to pod.
#### Create a sidecar proxy app <a id="create-a-sidecar-proxy-app"></a>
Create your .yaml
```bash
kubectl run app --image=bash --comand -oyaml --dry-run=client > <appName.yaml> -- shj -c 'ping google.com'
```
Edit your .yaml and add the uncomment lines:
```yaml
#apiVersion: v1
#kind: Pod
#metadata:
# name: security-context-demo
#spec:
# securityContext:
# runAsUser: 1000
# runAsGroup: 3000
# fsGroup: 2000
# volumes:
# - name: sec-ctx-vol
# emptyDir: {}
# containers:
# - name: sec-ctx-demo
# image: busybox
command: [ "sh", "-c", "apt update && apt install iptables -y && iptables -L && sleep 1h" ]
securityContext:
capabilities:
add: ["NET_ADMIN"]
# volumeMounts:
# - name: sec-ctx-vol
# mountPath: /data/demo
# securityContext:
# allowPrivilegeEscalation: true
```
See the logs of the proxy:
```bash
kubectl logs app -C proxy
```
More info at: [https://kubernetes.io/docs/tasks/configure-pod-container/security-context/](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/)
## Search vulnerable network services
As you are inside the Kubernetes environment, if you cannot escalate privileges abusing the current pods privileges and you cannot escape from the container, you should **search potential vulnerable services.**

View File

@ -89,8 +89,6 @@ So just create the malicious pod and expect the secrets in port 6666:
![](../../.gitbook/assets/image%20%28470%29.png)
### \*\*\*\*
## **Create/Update Deployment, Daemonsets, Statefulsets, Replicationcontrollers, Replicasets, Jobs and Cronjobs**
Deployment, Daemonsets, Statefulsets, Replicationcontrollers, Replicasets, Jobs and Cronjobs are all privileges that allow the creation of different tasks in the cluster. Moreover, it's possible can use all of them to **develop pods and even create pods**. So it's possible to a**buse them to escalate privileges just like in the previous example.**