diff --git a/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/certifications.md b/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/certifications.md new file mode 100644 index 000000000..e6203085c --- /dev/null +++ b/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/certifications.md @@ -0,0 +1,116 @@ +--- +docType: "Chapter" +id: "certifications" +chapterTitle: "Certifications" +description: "Get an overview of the existing Kubernetes certifications and what you need to learn for the CKA." +lectures: 10 +title: "Certifications" +weight: 1 +--- + +{{< chapterstyle >}} + +Get an overview of the existing Kubernetes certifications and what you need to learn for the CKA. + +## Several certifications available +--- + +| Certification | Type | Badge | +|---------------|------|-------| +| Kubernetes and Cloud Native Associate (KCNA) | MCQ | | +| Kubernetes and Cloud Native Security Associate (KCSA) | MCQ | | +| Certified Kubernetes Application Developer (CKAD) | Practice | | +| Certified Kubernetes Administrator (CKA) | Practice | | +| Certified Kubernetes Security Specialist (CKS)
\*passing the CKA is a requirement before passing the CKS* | Practice | | + + +If you pass all those certifications, you become a [Kubestronaut](https://www.cncf.io/training/kubestronaut/). + +## Expectation for the CKA +--- + +The following table summarizes the distribution of the CKA questions across 5 main subjects. + +| Subject | % | +|---------|---| +| Cluster Architecture, Installation & Configuration | 25% | +| Workloads & Scheduling | 15% | +| Services & Networking | 20% | +| Storage | 10% | +| Troubleshooting | 30% | + +## CKA Environment +--- + +The CKA is a 2h exam. It contains 15/20 questions and requires at least 66% correct answers. This exam is remotely proctored, so you can take it from home (or any other quiet location) at a time that best suits your schedule. + +Before launching the exam, which you do via your [Linux Foundation Training Portal](https://trainingportal.linuxfoundation.org/access/saml/login), you need to perform a couple of prerequisites including making sure the PSI Browser works correctly on your environment. This browser gives you access to the remote Desktop you'll use during the exam. + +{{< image src="/images/learning-path/cka/certifications/psi-browser.png" width="100%" align="center" alt="" >}} + +## Tips & tricks +--- + +### Tools + +Make sure you have a basic knowledge of + +- **vim** +- **openssl** + +```bash +# Visualize the content of a certificate +openssl x509 -in cert.crt -noout -text +``` + +- **systemd / systemctl / journalctl** + +```bash +# Restart kubelet +systemctl restart kubelet + +# Check kubelet logs +journalctl -u kubelet +``` + +### Aliases + +Defining a couple of aliases at the very beginning of the examination could save time. + +```bash +alias k=kubectl +export dr="--dry-run=client -o yaml" +export fd="--grace-period=0 --force" +``` + +### Imperative commands + +Don't create specifications manually, instead use `--dry-run=client -o yaml` as in these examples. + +```bash +k run nginx --image=nginx:1.20 --dry-run=client -o yaml > pod.yaml +k create deploy www --image=nginx:1.20 --replicas=3 --dry-run=client -o yaml > deploy.yaml +k create role create-pod --verb=create --resource=pods --dry-run=client -o yaml > role.yaml +``` + +Quickly change the current Namespace. + +```bash +k config set-context --current --namespace=dev +``` + +Don't wait for the grace period to get rid of a Pod. + +```bash +k delete po nginx --force --grace-period=0 +``` + +### Reference guide + +The [Kubectl quick reference guide](https://kubernetes.io/docs/reference/kubectl/quick-reference/) is a must-read. + +### Access to exam simulator + +Registering for the CKA gives you access to two sessions of the official Exam simulator. I highly recommend using these sessions once you're almost ready. + +{{< /chapterstyle >}} \ No newline at end of file diff --git a/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/creation.md b/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/creation.md new file mode 100644 index 000000000..5b5722c68 --- /dev/null +++ b/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/creation.md @@ -0,0 +1,219 @@ +--- +docType: "Chapter" +id: "creation" +chapterTitle: "Create a cluster" +description: "Build a 3-node kubeadm cluster from scratch." +lectures: 10 +title: "Create a cluster" +weight: 2 +--- + +{{< chapterstyle >}} + +This section guides you in creating of a 3-nodes Kubernetes cluster using [kubeadm](https://kubernetes.io/docs/reference/setup-tools/kubeadm/) bootstrapping tool. This is an important step as you will use this cluster throughout this workshop. + +The cluster you'll create is composed of 3 Nodes named **controlplane**, **worker1** and **worker2**. The controlplane Node runs the cluster components (API Server, Controller Manager, Scheduler, etcd), while worker1 and worker2 are the worker Nodes in charge of running the containerized workloads. + +{{< image src="/images/learning-path/cka/creation/objectives.png" width="100%" align="center" alt="" >}} + +## Provisioning VMs +--- + +Before creating a cluster, it's necessary to provision the infrastructure (bare metal servers or virtual machines). You can create the 3 VMs on your local machine or a cloud provider (but this last option will come with a small cost). Ensure you name those VMs **controlplane**, **worker1**, and **worker2** to keep consistency alongside the workshop. Please also ensure each VM has at least 2 vCPUs and 2G of RAM so it meets the [prerequisites](https://bit.ly/kubeadm-prerequisites). + +If you want to create those VMs on your local machine, we recommend using [Multipass](https://multipass.run), a tool from [Canonical](https://canonical.com/). Multipass makes creating local VMs a breeze. Once you have installed Multipass, create the VMs as follows. + +```bash +multipass launch --name controlplane --memory 2G --cpus 2 --disk 10G +multipass launch --name worker1 --memory 2G --cpus 2 --disk 10G +multipass launch --name worker2 --memory 2G --cpus 2 --disk 10G +``` + +{{< image src="/images/learning-path/cka/creation/step-1.png" width="100%" align="center" alt="" >}} + +## Cluster initialization +--- + +Now that the VMs are created, you need to install some dependencies on each on them (a couple of packages including **kubectl**, **containerd** and **kubeadm**). To simplify this process we provide some scripts that will do this job for you. + +First, ssh on the controlplane VM and install those dependencies using the following command. + +```bash +curl https://luc.run/kubeadm/controlplane.sh | VERSION="1.32" sh +``` + +Next, still from the controlplane VM, initialize the cluster. + +```bash +sudo kubeadm init +``` + +The initialization should take a few tens of seconds. The list below shows all the steps it takes. + +``` +preflight Run pre-flight checks +certs Certificate generation + /ca Generate the self-signed Kubernetes CA to provision identities for other Kubernetes components + /apiserver Generate the certificate for serving the Kubernetes API + /apiserver-kubelet-client Generate the certificate for the API server to connect to kubelet + /front-proxy-ca Generate the self-signed CA to provision identities for front proxy + /front-proxy-client Generate the certificate for the front proxy client + /etcd-ca Generate the self-signed CA to provision identities for etcd + /etcd-server Generate the certificate for serving etcd + /etcd-peer Generate the certificate for etcd nodes to communicate with each other + /etcd-healthcheck-client Generate the certificate for liveness probes to healthcheck etcd + /apiserver-etcd-client Generate the certificate the apiserver uses to access etcd + /sa Generate a private key for signing service account tokens along with its public key +kubeconfig Generate all kubeconfig files necessary to establish the control plane and the admin kubeconfig file + /admin Generate a kubeconfig file for the admin to use and for kubeadm itself + /super-admin Generate a kubeconfig file for the super-admin + /kubelet Generate a kubeconfig file for the kubelet to use *only* for cluster bootstrapping purposes + /controller-manager Generate a kubeconfig file for the controller manager to use + /scheduler Generate a kubeconfig file for the scheduler to use +etcd Generate static Pod manifest file for local etcd + /local Generate the static Pod manifest file for a local, single-node local etcd instance +control-plane Generate all static Pod manifest files necessary to establish the control plane + /apiserver Generates the kube-apiserver static Pod manifest + /controller-manager Generates the kube-controller-manager static Pod manifest + /scheduler Generates the kube-scheduler static Pod manifest +kubelet-start Write kubelet settings and (re)start the kubelet +upload-config Upload the kubeadm and kubelet configuration to a ConfigMap + /kubeadm Upload the kubeadm ClusterConfiguration to a ConfigMap + /kubelet Upload the kubelet component config to a ConfigMap +upload-certs Upload certificates to kubeadm-certs +mark-control-plane Mark a node as a control-plane +bootstrap-token Generates bootstrap tokens used to join a node to a cluster +kubelet-finalize Updates settings relevant to the kubelet after TLS bootstrap + /enable-client-cert-rotation Enable kubelet client certificate rotation +addon Install required addons for passing conformance tests + /coredns Install the CoreDNS addon to a Kubernetes cluster + /kube-proxy Install the kube-proxy addon to a Kubernetes cluster +show-join-command Show the join command for control-plane and worker node +``` + +Several commands are returned at the end of the installation process, which you'll use in the next part. + +{{< image src="/images/learning-path/cka/creation/step-2.png" width="100%" align="center" alt="" >}} + +## Retrieving kubeconfig file +--- + +The first set of commands returned during the initialization step allows configuring kubectl for the current user. Run those commands from a shell in the controlplane Node. + +```bash +mkdir -p $HOME/.kube +sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config +sudo chown $(id -u):$(id -g) $HOME/.kube/config +``` + +You can now list the Nodes. You'll get only one Node as you've not added the worker Nodes yet. + +```bash +$ kubectl get no +NAME STATUS ROLES AGE VERSION +controlplane NotReady control-plane 5m4s v1.32.4 +``` + +## Adding the first worker Node +--- + +As you've done for the controlplane, use the following command to install the dependencies (kubectl, containerd, kubeadm) on worker1. + +```bash +curl https://luc.run/kubeadm/worker.sh | VERSION="1.32" sh +``` + +Then, run the join command returned during the initialization step. This command allows you to add worker nodes to the cluster. + +```bash +sudo kubeadm join 10.81.0.174:6443 --token kolibl.0oieughn4y03zvm7 \ + --discovery-token-ca-cert-hash sha256:a1d26efca219428731be6b62e3298a2e5014d829e51185e804f2f614b70d933d +``` + +## Adding the second worker Node +--- + +You need to do the same on worker2. First, install the dependencies. + +```bash +curl https://luc.run/kubeadm/worker.sh | VERSION="1.32" sh +``` + +Then, run the join command to add this Node to the cluster. + +```bash +sudo kubeadm join 10.81.0.174:6443 --token kolibl.0oieughn4y03zvm7 \ + --discovery-token-ca-cert-hash sha256:a1d26efca219428731be6b62e3298a2e5014d829e51185e804f2f614b70d933d +``` + +You now have cluster with 3 Nodes. + +{{< image src="/images/learning-path/cka/creation/step-3.png" width="100%" align="center" alt="" >}} + +## Status of the Nodes +--- + +List the Nodes and notice they are all in NotReady status. + +```bash +$ kubectl get nodes +NAME STATUS ROLES AGE VERSION +controlplane NotReady control-plane 9m58s v1.32.4 +worker1 NotReady 58s v1.32.4 +worker2 NotReady 55s v1.32.4 +``` + +If you go one step further and describe the controlplane Node, you'll get why the cluster is not ready yet. + +``` +… +KubeletNotReady container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized +``` + +## Installing a network plugin +--- + +Run the following commands from the controlplane Node to install Cilium in your cluster. + +```bash +OS="$(uname | tr '[:upper:]' '[:lower:]')" +ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')" +curl -L --remote-name-all https://github.com/cilium/cilium-cli/releases/latest/download/cilium-$OS-$ARCH.tar.gz{,.sha256sum} +sudo tar xzvfC cilium-$OS-$ARCH.tar.gz /usr/local/bin +cilium install +``` + +After a few tens of seconds, you'll see your cluster is ready. + +```bash +$ kubectl get nodes +NAME STATUS ROLES AGE VERSION +controlplane Ready control-plane 13m v1.32.4 +worker1 Ready 4m28s v1.32.4 +worker2 Ready 4m25s v1.32.4 +``` + +## Get the kubeconfig on the host machine +--- + +To avoid connecting to the controlplane Node to run the kubectl commands, copy the kubeconfig file from the controlplane to the host machine. Make sure to copy this file into `$HOME/.kube/config` so it automatically configures kubectl. + +If you've created your VMs with Multipass, you can copy the kubeconfig file using the following commands. + +```bash +multipass transfer controlplane:/home/ubuntu/.kube/config config +mkdir $HOME/.kube +mv config $HOME/.kube/config +``` + +You should now be able to direcly list the Nodes from the host machine. + +```bash +$ kubectl get nodes +NAME STATUS ROLES AGE VERSION +controlplane Ready control-plane 13m v1.32.4 +worker1 Ready 4m28s v1.32.4 +worker2 Ready 4m25s v1.32.4 +``` + +{{< /chapterstyle >}} \ No newline at end of file diff --git a/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/networking.md b/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/networking.md new file mode 100644 index 000000000..942ca1c85 --- /dev/null +++ b/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/networking.md @@ -0,0 +1,222 @@ +--- +docType: "Chapter" +id: "networking" +chapterTitle: "Networking" +description: "Understand Pod to Pod communication, Service discovery, Ingress resources." +lectures: 10 +title: "Networking" +weight: 5 +--- + +{{< chapterstyle >}} + +This section is a refresher that provides an overview of the main Kubernetes resources related to networking. At the end of this section, please complete the exercises to put these concepts into practice. + +## Kubernetes Networking Model +--- + +In Kubernetes, a network plugin ensures communication between the Pods. Each network plugin must implement the following requirements: + +- all Pods can communicate with all other Pods without NAT +- all Nodes can communicate with all Pods without NAT +- the IP that a Pod sees itself as is the same IP that others see it as + +## Communication Types +--- + +There are different types of communication within a cluster: + +- **Container to container** +- **Pod to Pod** + - on the same Node + - on different Nodes +- **Pod to Service** +- **External to Service** + - NodePort service + - LoadBalancer Service + - Ingress Controller + +### **Container to Container Communication** + +When a Pod is created, it has its own network namespace which is set up by a **pause container**. This container is special as it does not run any workload and is not visible from the kubectl commands. The other containers of this Pod are all attached to the pause container's network namespace and are thus communicating through `localhost`. + +{{< image src="/images/learning-path/cka/networking/container-to-container.png" width="100%" align="center" alt="" >}} + +### **Pod to Pod on the Same Node** + +Each Pod has its own network namespace and they communicate via a virtual Ethernet (veth) pair connected to a bridge on the host. This setup allows Pod-to-Pod traffic to be switched locally without leaving the Node. + +{{< image src="/images/learning-path/cka/networking/pod-to-pod-same-node.png" width="100%" align="center" alt="" >}} + +### **Pod to Pod Across Different Nodes** + +The network plugin ensures that each Pod's IP is routable across the cluster, using encapsulation, overlays, or native routing. Packets travel across the network infrastructure between Nodes before reaching the destination Pod's virtual interface. + +{{< image src="/images/learning-path/cka/networking/pod-to-pod-different-nodes.png" width="100%" align="center" alt="" >}} + +## Network Plugin +--- + +A Network plugin is mandatory in a Kubernetes cluster. It ensures the communication between Pods across the cluster, whether they are on the same Node or on different Nodes. + +Among the network plugins available, **Kubenet** is a basic and simple one, it has a limited set of functionalities and cannot be used with kubeadm. All other major plugins implement the Container Networking Interface (CNI) specification. + +- [https://github.com/containernetworking](https://github.com/containernetworking) - CNCF incubated project +- [https://cncf.io/projects](https://cncf.io/projects) - Manages containers' network connectivity +- More info: [https://bit.ly/about-cni-plugins](https://bit.ly/about-cni-plugins) + +{{< image src="/images/learning-path/cka/networking/cni.png" width="100%" align="center" alt="" >}} + +## Container Network Interface (CNI) +--- + +CNI provides a standard interface for configuring network interfaces in Linux containers, allowing plugins to implement their own advanced functionalities such as routing, network security, and more. + +Popular CNI plugins include: + +- [Cilium](https://cilium.io) - advanced security, eBPF-based +- [Calico](https://www.tigera.io/project-calico/) - policy engine, supports BGP +- [Flannel](https://github.com/flannel-io/flannel) - simple, uses VXLAN by default + +These plugins are typically: + +- Installed as a DaemonSet +- Designed for different use cases (low latency, enhanced security, observability, etc.) +- Can use encapsulation (L2/VXLAN) or non-encapsulated (L3/BGP) modes depending on your setup and requirements + +### Communication Types + +#### Encapsulated (VXLAN) + +Traffic between Pods is wrapped (encapsulated) in another packet and routed through an overlay network. + +{{< image src="/images/learning-path/cka/networking/encapsulated.png" width="100%" align="center" alt="" >}} + +#### Unencapsulated (BGP) + +Traffic is routed directly between nodes without encapsulation, using protocols like BGP to advertise Pod networks. + +{{< image src="/images/learning-path/cka/networking/uncapsulated.png" width="100%" align="center" alt="" >}} + +## Service +--- + +A Service is a Kubernetes resource that provides a stable networking endpoint to expose a group of Pods. Services ensure that communication to a group of Pods is reliable, even as Pods are dynamically created or destroyed. + +Main types of Services: + +- **ClusterIP (default):** exposes the Service internally within the cluster. Not accessible from outside +- **NodePort:** exposes the Service on a static port on each Node of the cluster +- **LoadBalancer:** creates an external load balancer (only available when using a cloud provider) to expose the Service to the internet + +Key Characteristics: + +- Each Service is assigned a Virtual IP (VIP), which stays the same during the lifecycle of the Service +- Services use labels and selectors to dynamically group Pods +- kube-proxy configures network rules on Nodes to route traffic from the Service to the appropriate Pods + +### Service of Type ClusterIP + +A Service of type ClusterIP exposes a group of Pods inside the cluster, so that other Pods can reach them. + +{{< image src="/images/learning-path/cka/networking/ClusterIP.png" width="100%" align="center" alt="" >}} + +### Service of Type NodePort + +A Service of type NodePort exposes a group of Pods to the external world, opening the same port on each Node. + +{{< image src="/images/learning-path/cka/networking/NodePort.png" width="100%" align="center" alt="" >}} + +### Service of Type LoadBalancer + +A Service of type LoadBalancer exposes a group of Pods to the external world through a load balancer. This feature is only available for clusters running on cloud providers. + +{{< image src="/images/learning-path/cka/networking/LoadBalancer.png" width="100%" align="center" alt="" >}} + +## Endpoints +--- + +Endpoints resources are the list of IP:PORT of the pods exposed by a Service. Endpoints are updated each time a Pod is created/updated. The commands below create a Deployment with 3 Pods and expose them with a Service. + +```bash +Creation of a deployment +kubectl create deploy ghost --replicas=3 --image=ghost:4 + +Exposition through a service +kubectl expose deploy/ghost --port=2368 +``` + +We can query the Endpoints directly. + +```bash +$ kubectl get endpoints ghost +NAME ENDPOINTS AGE +ghost 10.0.0.210:2368,10.0.0.25:2368,10.0.1.252:2368 51s +``` + +Or, get the list of Endpoints from the Service's details. + +```bash +$ kubectl describe svc ghost +Name: ghost +Namespace: default +Labels: app=ghost +Annotations: +Selector: app=ghost +Type: ClusterIP +IP Family Policy: SingleStack +IP Families: IPv4 +IP: 10.107.128.221 +IPs: 10.107.128.221 +Port: 2368/TCP +TargetPort: 2368/TCP +Endpoints: 10.0.1.252:2368,10.0.0.210:2368,10.0.0.25:2368 +Session Affinity: None +Internal Traffic Policy: Cluster +Events: +``` + +## Pod to Service Communication +--- + +By default, **kube-proxy** sets the network rules when we create a Service. These rules allow access to the backend Pods when accessing the Service. + +kube-proxy currently uses **iptables** as the default data-plane mode, but another mode can be enabled instead, such as **IPVS** or **eBPF** (via Calico or Cilium network plugins). + +This [article](https://bit.ly/kube-proxy-iptables) provides additional information about how the iptables rules are created. + +## Ingress Controller +--- + +An Ingress Controller is a reverse proxy exposing ClusterIP services to the outside. It's usually the single entry point exposed by an external load balancer. + +{{< image src="/images/learning-path/cka/networking/ingress.png" width="100%" align="center" alt="" >}} + +An Ingress Controller is configured with resources of type Ingress which allows L7 routing via the domain name or a path within the URL. Below is an example of Ingress specification. It redirects all traffic targeting the `/api` endpoint to the `api` Service listening on port 80. + +```yaml +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: ingress + annotations: + nginx.ingress.kubernetes.io/rewrite-target: / +spec: + ingressClassName: nginx + rules: + - http: + paths: + - path: /api + pathType: Prefix + backend: + service: + name: api + port: + number: 80 +``` + +## Practice +--- + +You can now jump to the [Exercises part](./exercises/) to learn and practice the concepts above. +{{< /chapterstyle >}} \ No newline at end of file diff --git a/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/operations.md b/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/operations.md new file mode 100644 index 000000000..29b9793f9 --- /dev/null +++ b/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/operations.md @@ -0,0 +1,143 @@ +--- +docType: "Chapter" +id: "operations" +chapterTitle: "Operations" +description: "Perform cluster upgrade and backup/restore of etcd." +lectures: 10 +title: "Operations" +weight: 9 +--- + +{{< chapterstyle >}} + +This section details some Day-2 operations, the main ones being the cluster upgrade and etcd backup/restore. At the end of this section, please complete the exercises in the order provided. + +## Process to upgrade a kubeadm cluster +--- + +Upgrading a kubeadm cluster is a very well-documented process. First, we upgrade the control plane Nodes, one at a time, following these steps: + +- change package repository +- get a version to upgrade to +- upgrade kubeadm binary +- run the upgrade +- drain the Node +- upgrade kubelet & kubectl +- uncordon the Node + +Next, we upgrade the worker Nodes, one at the time, following these steps: + +- change package repository +- upgrade kubeadm binary +- run the upgrade +- drain the Node +- upgrade kubelet +- uncordon the Node + +## About etcd +--- + +[etcd](https://etcd.io) is a distributed key-value store. It is considered the brain of Kubernetes as it contains all the information about the cluster's resources. + +An highly available Kubernetes cluster requires multiple etcd instances. In that case the RAFT consensus algorithm maintains consistency between these instances. + +[The Secret Live Of Data](http://thesecretlivesofdata.com) is a great resource to understand how RAFT works. + +## Communicating with etcd +--- + +From the control plane Node, we can communicate with etcd using the etcdctl utility. This binary is installed on your Node. + +The example below gets the etcd status. + +```bash +sudo ETCDCTL_API=3 etcdctl \ +--endpoints localhost:2379 \ +--cert=/etc/kubernetes/pki/apiserver-etcd-client.crt \ +--key=/etc/kubernetes/pki/apiserver-etcd-client.key \ +--cacert=/etc/kubernetes/pki/etcd/ca.crt \ +endpoint health +``` + +This one lists the members of the etcd cluster. + +```bash +sudo ETCDCTL_API=3 etcdctl \ +--endpoints localhost:2379 \ +--cert=/etc/kubernetes/pki/apiserver-etcd-client.crt \ +--key=/etc/kubernetes/pki/apiserver-etcd-client.key \ +--cacert=/etc/kubernetes/pki/etcd/ca.crt \ +member list +``` + +## Creating a backup +--- + +To back up etcd, we need to run the following command which creates the file snapshot.db in the current folder. + +```bash +$ sudo ETCDCTL_API=3 etcdctl snapshot save \ +--endpoints localhost:2379 \ +--cacert /etc/kubernetes/pki/apiserver-etcd-client.crt \ +--cert /etc/kubernetes/pki/apiserver-etcd-client.key \ +--key /etc/kubernetes/pki/etcd/ca.key \ +snapshot.db +``` + +Next, we verify the backup. + +```bash +$ sudo ETCDCTL_API=3 etcdctl \ +--write-out=table snapshot status snapshot.db +``` + +## Restore +--- + +The restoration process requires several steps. + +First, we restore a backup in the `/var/lib/etcd-snapshot` folder, this one has to be different from the one used by default, which is `/var/lib/etcd`. + +```bash +$ sudo ETCDCTL_API=3 etcdctl snapshot restore \ +--endpoints localhost:2379 \ +--cacert /etc/kubernetes/pki/etcd/server.crt \ +--key /etc/kubernetes/pki/apiserver-etcd-client.crt \ +--cert /etc/kubernetes/pki/apiserver-etcd-client.key \ +--data-dir /var/lib/etcd-snapshot \ +snapshot.db +``` + +Next, we stop the API Server. This step is important as we don't want the API Server to mess up while etcd is not available. As the API Server is a static Pod, directly managed by kubelet, we just need to move its specification to another folder so that kubelet delete the Pod. + +```bash +sudo mv /etc/kubernetes/manifests/kube-apiserver.yaml /etc/kubernetes/ +``` + +Next, we update the etcd manifests, specifying the folder from which it needs to get its data. + +```yaml +… +volumes: +- hostPath: + path: /etc/kubernetes/pki/etcd + type: DirectoryOrCreate + name: etcd-certs +- hostPath: + path: /var/lib/etcd-snapshot <- New location of data + type: DirectoryOrCreate + name: etcd-data +``` + +Then, once etcd is running, we restart the API Server, moving its specification back to the `/etc/kubernetes/manifests` folder. + +```bash +sudo mv /etc/kubernetes/kube-apiserver.yaml /etc/kubernetes/manifests/ +``` + +## — Practice — +--- + +You can now jump to the [Exercises part](./exercises/) to perform these Day-2 operations and a couple of other manipulations on your cluster. + +{{< /chapterstyle >}} \ No newline at end of file diff --git a/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/scheduling.md b/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/scheduling.md new file mode 100644 index 000000000..135d1c0ae --- /dev/null +++ b/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/scheduling.md @@ -0,0 +1,331 @@ +--- +docType: "Chapter" +id: "scheduling" +chapterTitle: "Scheduling Pods" +description: "Learn Pod placement strategies using labels, taints, affinities and more." +lectures: 10 +title: "Scheduling Pods" +weight: 4 +--- + +{{< chapterstyle >}} + +This section is a refresher that provides an overview of the main properties involved in the scheduling phase. At the end of this section, please complete the exercises to apply these concepts. + +## Purpose +--- + +The scheduling step is where Kubernetes decides on which Node a Pod will run on. The Scheduler, running on the control plane, is the process in charge of this action. + +{{< image src="/images/learning-path/cka/scheduling/scheduling.png" width="100%" align="center" alt="" >}} + +There are various properties/items which can influence the scheduling decision, including: + +- `nodeName` +- `nodeSelector` +- `nodeAffinity` +- `podAffinity / podAntiAffinity` +- `topologySpreadConstraints` +- `taint / toleration` +- `available resources` +- `priorityClass` +- `runtimeClass` + +## nodeName +--- + +The `nodeName` property bypasses the scheduling process, indicating directly in the Pod's specification the name of the Node this Pod must be deployed to. + +{{< image src="/images/learning-path/cka/scheduling/nodeName.png" width="100%" align="center" alt="" >}} + +## nodeSelector +--- + +The `nodeSelector` property uses Node's labels to schedule a Pod. + +{{< image src="/images/learning-path/cka/scheduling/nodeSelector.png" width="100%" align="center" alt="" >}} + +## nodeAffinity +--- + +The `nodeAffinity` property also uses Node's label. Still, it is more granular than `nodeSelector` as it can use the following operators on the labels: **In**, **NotIn**, **Exists**, **DoesNotExist**, **Gt**, and **Lt**. + +Two rules are available when using `nodeAffinity`: + +- `requiredDuringSchedulingIgnoredDuringExecution` defines a **hard constraint**: if the scheduler does not manage to schedule the Pod according to the specification, then the Pod will remain in Pending +- `preferredDuringSchedulingIgnoredDuringExecution` defines a **soft constraint**: if the scheduler does not manage to schedule the Pod, according to the specification, then it will do its best to schedule it anyway, even if the requirements are not satisfied + +### Example: + +```yaml +spec: + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: kubernetes.io/e2e-az-name Pod must be scheduled on a node having the label + operator: In kubernetes.io/e2e-az-name with the value e2e-az1 or e2e-az2 + values: + - e2e-az1 + - e2e-az2 + preferredDuringSchedulingIgnoredDuringExecution: + - weight: 1 + preference: + matchExpressions: Pod preferably scheduled on a node having the label + - key: disktype disktype with the value ssd + operator: In + values: + - ssd +``` + +## podAffinity / podAntiAffinity +--- + +We use the `podAffinity` and `podAntiAffinity` properties to schedule a Pod based on the labels of already existing Pods. + +It uses the same rules as the `nodeAffinity` property: +- `requiredDuringSchedulingIgnoredDuringExecution` defines a **hard constraint** +- `preferredDuringSchedulingIgnoredDuringExecution` defines a **soft constraint** + +It also uses a property named `topologyKey` to specify geographical preferences (among other things): +- `hostname` +- `region` +- `az` +- ... + +### Example 1 + +The following Deployment's Pods cannot all be created on a cluster with less than 4 Nodes as it specifies that two Pods with the `app: cache` label must not be on the same Node. + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: cache +spec: + replicas: 4 + selector: + matchLabels: + app: cache + template: + metadata: + labels: + app: cache + spec: + containers: + - name: redis + image: redis:6 + affinity: + podAntiAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - labelSelector: + matchExpressions: + - key: app + operator: In + values: + - cache + topologyKey: "kubernetes.io/hostname" +``` + +### Example 2 + +The following specification illustrates the usage of both `podAffinity` and `podAntiAffinity` properties. + +```yaml +spec: + affinity: + podAffinity: + requiredDuringSchedulingIgnoredDuringExecution: Pod must be scheduled on a node which is in the availability + - labelSelector: zone where already exists a Pod with the label security: S1 + matchExpressions: + - key: security + operator: In + values: + - S1 + topologyKey: failure-domain.beta.kubernetes.io/zone + podAntiAffinity: + preferredDuringSchedulingIgnoredDuringExecution: Pod should not be scheduled on a node where already exists + - podAffinityTerm: a Pod with the label security: S2 + labelSelector: + matchExpressions: + - key: security + operator: In + values: + - S2 + topologyKey: kubernetes.io/hostname +``` + +## TopologySpreadConstraints +--- + +The `topologySpreadConstraints` property defines how to spread Pods across a cluster topology, ensuring application resiliency. + +The following Deployment uses the `topologySpreadConstraints` property to ensure Pods are correctly balanced between AZ and Node. + +{{< image src="/images/learning-path/cka/scheduling/topologySpreadConstraint.png" width="100%" align="center" alt="" >}} + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: www +spec: + replicas: 4 + selector: + matchLabels: + app: www + template: + metadata: + labels: + app: www + spec: + containers: + - name: nginx + image: nginx:1.24 + topologySpreadConstraints: + - maxSkew: 1 The difference in the number of matching Pods between + topologyKey: "kubernetes.io/hostname" any two nodes should be no more than 1 (maxSkew) + whenUnsatisfiable: ScheduleAnyway + labelSelector: + matchLabels: + app: www + - maxSkew: 1 Ensures that the Pods are spread across different + topologyKey: "topology.kubernetes.io/zone" availability zones + whenUnsatisfiable: ScheduleAnyway + labelSelector: + matchLabels: + app: www +``` + +## Taints & Tolerations +--- + +In contrast to the properties we saw earlier, we use a **Taint** to prevent certain Pods from being scheduled on a node. A Pod must tolerate a Taint, using a `toleration` property, to be scheduled on a Node having that Taint. + +A Taint has 3 properties: +- `key`: it can be arbitrary string content, same format as labels +- `value`: it can be an arbitrary string content, same format as labels +- `effect`: among `NoSchedule`, `PreferNoSchedule` and `NoExecute` + +By default, a kubeadm cluster sets a Taint on the control plane Nodes. This Taint prevents a Pod from being deployed on these Nodes unless the Pod explicitly tolerates this Taint. + +The following command lists the Taints existing on the controlplane Node. + +```bash +$ kubectl get no controlplane -o jsonpath='{.spec.taints}' | jq +[ + { + "effect": "NoSchedule", + "key": "node-role.kubernetes.io/control-plane" + } +] +``` + +When creating the following Deployment, with 10 replicas of nginx Pods, none of the Pods will land on the controlplane because of this specific Taint. + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: www +spec: + replicas: 10 + selector: + matchLabels: + app: www + template: + metadata: + labels: + app: www + spec: + containers: + - image: nginx:1.24 + name: nginx +``` + +We must add a toleration for the Taint, so the scheduler can schedule Pods on the controlplane. + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: www +spec: + replicas: 10 + selector: + matchLabels: + app: www + template: + metadata: + labels: + app: www + spec: + tolerations: + - key: node-role.kubernetes.io/control-plane + effect: NoSchedule + containers: + - image: nginx:1.24 + name: nginx +``` + +## PriorityClass +--- + +A `PriorityClass` is a Kubernetes resource that defines the priority of Pods. It influences scheduling decisions and preemption - meaning that higher-priority Pods can evict lower-priority ones if resources are scarce. A PriorityClass can be preempting (default behavior) or non-preempting, depending on the `preemtionPolicy` property. + +The following specifications define a high-priority PriorityClass and a Pod using it. + +```yaml +apiVersion: scheduling.k8s.io/v1 +kind: PriorityClass +metadata: + name: high-priority +value: 1000000 +globalDefault: false +--- +apiVersion: v1 +kind: Pod +metadata: + name: nginx +spec: + containers: + - name: nginx + image: nginx + priorityClassName: high-priority +``` + +## RuntimeClass +--- + +A `RuntimeClass` allows to have multiple container runtimes in the cluster, and to select the one which best fits a workload. There are several container runtimes, each one of them addresses specific use cases, including: + +- [Containerd](https://containerd.io/) is a general-purpose container runtime (CNCF Graduated) +- [CRI-O](https://github.com/cri-o/cri-o) is a light container runtime focusing on Kubernetes (CNCF Graduated) +- [GVisor](https://gvisor.dev/) is used to run an unsecure workload in a sandbox +- [Kata-Container](https://katacontainers.io/) uses Micro VMs for workload isolation +- [Firecracker](https://firecracker-microvm.github.io/) uses Micro VMs, mainly for serverless +- [WasmEdge](https://wasmedge.org/) is dedicated to run Wasm workload + +In a Pod specification, we can specify the container runtime configuration needed by this workload. The scheduler ensures that Pods are placed on Nodes that support the required container runtime configuration. + +Let's say our cluster already has a RuntimeClass named `gvisor`, then we can use it in the Pod specification as follows. + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: nginx +spec: + containers: + - name: nginx + image: nginx + runtimeClass: gvisor +``` + +## Practice +--- + +You can now jump to the [Exercises part](./exercises/) to learn and practice the concepts above. + +{{< /chapterstyle >}} \ No newline at end of file diff --git a/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/security.md b/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/security.md new file mode 100644 index 000000000..d719e7920 --- /dev/null +++ b/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/security.md @@ -0,0 +1,368 @@ +--- +docType: "Chapter" +id: "security" +chapterTitle: "Security" +description: "Create Network Policies and RBAC rules." +lectures: 10 +title: "Security" +weight: 7 +--- + +{{< chapterstyle >}} + +This section is a refresher that provides an overview of the main concepts of security in Kubernetes. At the end of this section, please complete the exercises to put these concepts into practice. + +## Authentication & Authorization +--- + +{{< image src="/images/learning-path/cka/security/rbac.png" width="100%" align="center" alt="" >}} + +### Authentication: several methods + +Kubernetes offers multiple methods to authenticate users against the API Server: + +- Client certificates +- Bearer tokens +- HTTP basic auth +- OpenID Connect +- Proxy + +To authenticate an application running in a Pod, Kubernetes relies on ServiceAccounts resources. + +### Authentication: admin kubeconfig + +When we create a cluster, an admin kubeconfig file is generated, similar to the following one. + +```yaml +apiVersion: v1 +clusters: +- cluster: + certificate-authority-data: LS0tLS1CR…0tLS0tCg== + server: https://10.55.133.216:6443 + name: kubernetes +contexts: +- context: + cluster: kubernetes + user: kubernetes-admin + name: kubernetes-admin@kubernetes +current-context: kubernetes-admin@kubernetes +kind: Config +preferences: {} +users: +- name: kubernetes-admin + user: + client-certificate-data: LS0tLS1CRU...0tLS0tCg== + client-key-data: LS0tLS1CRU...0tLS0tCg== +``` + +This file contains a public/private key pair used for authentication against the API Server. We can use OpenSSL commands to get details about the public key (x509 certificate). + +{{< image src="/images/learning-path/cka/security/certificate.png" width="100%" align="center" alt="" >}} + +The following screenshot shows the Subject used in the certificate: + +- **"O = system:master"** indicates this certificate is related to the system:master group +- **"CN= kubernetes-admin"** indicates it is related to the kubernetes-admin user + +Using this certificate to communicate with the API Server will authenticate us as the kubernetes-admin belonging to the system:master group. This is a specific case, as the group system:master provides full access to the cluster. + +The admin kubeconfig file is not the only kubeconfig file generated during the cluster creation step. As we'll see in the next section, each component that needs to communicate with the API Server has its kubeconfig file (and associated access rights). + +### Authentication: control-plane components + +The following picture illustrates how the control plane components communicate with each other. + +{{< image src="/images/learning-path/cka/security/control-plane.png" width="100%" align="center" alt="" >}} + +The /etc/kubernetes folder contains the following files to ensure this communication is secured. + +- kubeconfig files to authenticate internal components against the API Server +- Certificates and private keys to ensure communication is using TLS; they are located in /etc/kubernetes/pki + +```bash +$ sudo tree /etc/kubernetes +/etc/kubernetes +├── admin.conf +├── controller-manager.conf +├── kubelet.conf +├── manifests +│ ├── etcd.yaml +│ ├── kube-apiserver.yaml +│ ├── kube-controller-manager.yaml +│ └── kube-scheduler.yaml +├── pki +│ ├── apiserver-etcd-client.crt +│ ├── apiserver-etcd-client.key +│ ├── apiserver-kubelet-client.crt +│ ├── apiserver-kubelet-client.key +│ ├── apiserver.crt +│ ├── apiserver.key +│ ├── ca.crt +│ ├── ca.key +│ ├── etcd +│ │ ├── ca.crt +│ │ ├── ca.key +│ │ ├── healthcheck-client.crt +│ │ ├── healthcheck-client.key +│ │ ├── peer.crt +│ │ ├── peer.key +│ │ ├── server.crt +│ │ └── server.key +│ ├── front-proxy-ca.crt +│ ├── front-proxy-ca.key +│ ├── front-proxy-client.crt +│ ├── front-proxy-client.key +│ ├── sa.key +│ └── sa.pub +├── scheduler.conf +└── super-admin.conf +``` + +For information purposes, the following table gives the subject of the certificates embedded in each kubeconfig file. + +| file | subject | +|------|---------| +| admin.conf | O = system:masters, CN=kubernetes-admin | +| super-admin.conf | O = system:masters, CN = kubernetes-super-admin | +| controller-manager.conf | CN = system:kube-controller-manager | +| kubelet.conf | O = system:nodes, CN = system:node:NODE_NAME | +| scheduler.conf | CN = system:kube-scheduler | + +### Authentication: ServiceAccount + +When a Pod needs to access the API Server, it must use a resource of type ServiceAccount. The following YAML specification defines a ServiceAccount named viewer. + +```yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: viewer +``` + +We can manually create a token for this ServiceAccount. + +```bash +kubectl create token viewer +``` + +This command returns a token similar to the following one: + +``` +eyJhbGciOiJSUzI1NiIsImtpZCI6IlRwSU85ZXdWUFp0SlpjaDBjekl6ZTNaNGRuUTZSVDFiV2dyWVhqbGwyRDAifQ.eyJhdWQiOlsiaHR0cHM6Ly9rdWJlcm5ldGVzLmRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwiXSwiZXhwIjoxNzQ1NDk5OTUyLCJpYXQiOjE3NDU0OTYzNTIsImlzcyI6Imh0dHBzOi8va3ViZXJuZXRlcy5kZWZhdWx0LnN2Yy5jbHVzdGVyLmxvY2FsIiwianRpIjoiMTE1OTgzZjYtOWE3Ny00ZmY1LWE4OGQtMTc2ODg3N2YxYmE3Iiwia3ViZXJuZXRlcy5pbyI6eyJuYW1lc3BhY2UiOiJkZWZhdWx0Iiwic2VydmljZWFjY291bnQiOnsibmFtZSI6InZpZXdlciIsInVpZCI6IjY2NmE3NWNkLWRkZGUtNDAzYi1iZmE0LWM0MjIxNWI1OTA1YiJ9fSwibmJmIjoxNzQ1NDk2MzUyLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGVmYXVsdDp2aWV3ZXIifQ.CGYbqWDj3KaEGPgU_pV6sL1wRf3IU56AlpljLxUO6tvpbkK7Z6le8FI5zdwp_04LgcWnHLo5-hsZiyJxmeKYXhsb3CASkI0Vvumfsb8kahIiJxVXIE-PfzKNlxampuubc3mG4q9h1s0M_Y-PubMdl4TkBoLMjujxbsTtPqpD2joxyZ2YB7ys7DiGp-BjQwXwwaxOniSwd0l_tyEAlX0UTy0qjmjjuMBJKQTLDzwPJXWCAXbeAMULsnsosS21sWyimmVMz6HQ8S4MttkMSg8eZ1IW-LPPn3Hfs0lBLRYeVRBn6qe4l7qxgCfgj57GfYgEWGy5BO9uaAAGcHVBdTacAQ +``` + +From [jwt.io](https://jwt.io) we can get the content of this JWT token and see that it authenticates the ServiceAccount named viewer within the default Namespace. We could use this token manually to call the API Server, but when a Pod is using a ServiceAccount a dedicated token is automatically created and mounted into its containers' filesystem. + +{{< image src="/images/learning-path/cka/security/jwt.png" width="100%" align="center" alt="" >}} + +## Authorization: RBAC (Role Based Access Control) +--- + +In the previous section, we covered the authentication mechanisms that allow the API Server to verify a user's or an application's identity. Now, we'll look at the resources used to grant permissions. + +{{< image src="/images/learning-path/cka/security/rbac-resources.png" width="100%" align="center" alt="" >}} + +### Authorization: Role / RoleBinding + +A Role resource defines permissions in a Namespace. A RoleBinding associates this Role with an identity which can be a user, a group, or a ServiceAccount. + +The following example defines a Role that grants read-only access to Pods in the development Namespace. The RoleBinding associates this Role with a user named bob: if a user has a certificate whose subject is bob, then he will be granted the permission to list and get information about Pods in the development Namespace. + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + namespace: development + name: dev-pod-reader +rules: +- apiGroups: [""] + resources: ["pods"] + verbs: ["get", "list"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: dev-pod-reader + namespace: development +subjects: +- kind: User + name: bob + apiGroup: rbac.authorization.k8s.io +roleRef: + kind: Role + name: dev-pod-reader + apiGroup: rbac.authorization.k8s.io +``` + +### Authorization: ClusterRole / ClusterRoleBinding + +The ClusterRole and ClusterRoleBinding resources are similar to the Role and RoleBinding ones, except that they are global to the cluster instead of being limited to a Namespace. + +The following specifications define a ClusterRole which grants read access to resources of type Secret in the entire cluster and a ClusterRoleBinding that associates this ClusterRole to the ServiceAccount named viewer. If a Pod uses the viewer ServiceAccount, then its containers will have the right to read the cluster's Secrets. + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: secret-reader +rules: +- apiGroups: [""] + resources: ["secrets"] + verbs: ["get", "list"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: dev-pod-reader +subjects: +- kind: ServiceAccount + name: viewer + apiGroup: rbac.authorization.k8s.io +roleRef: + kind: ClusterRole + name: secret-reader + apiGroup: rbac.authorization.k8s.io +``` + +### Authorization: usage of a ServiceAccount + +To use a ServiceAccount, we can define the serviceAccountName property in the Pod's specification as follows. + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: monitoring +spec: + containers: + - image: lucj/mon:1.2 + name: mon + serviceAccountName: viewer +``` + +## SecurityContext +--- + +The SecurityContext property defines privileges and access controls at the Pod or container level. + +When defined at the **Pod level**, we can use the following properties: + +- **fsGroup**: specifies a group ID that all containers in the Pod use for accessing mounted volumes +- **runAsGroup**: sets the primary group ID for all container processes +- **runAsNonRoot**: ensures that containers must run as a non-root user +- **runAsUser**: sets the user ID to run all processes in the Pod +- **seLinuxOptions**: defines SELinux labels to apply to the Pod +- **supplementalGroups**: specifies additional group IDs for processes in the Pod +- **sysctls**: allows setting kernel parameters (sysctls) for the Pod + +When defined at the **container level**, we can use the following properties: + +- **allowPrivilegeEscalation**: indicates whether a process can gain more privileges than its parent +- **capabilities**: adds or removes Linux capabilities for the container process +- **privileged**: grants the container full access to the host +- **procMount**: controls how /proc is mounted in the container +- **readOnlyRootFilesystem**: if set to true, makes the container's root filesystem read-only +- **runAsGroup**: sets the primary group ID for the container process +- **runAsNonRoot**: ensures the container runs as a non-root user +- **runAsUser**: sets the user ID to run the container process +- **seLinuxOptions**: applies SELinux labels to the container + +### Example + +The following Pod specification defines a securityContext property both at the Pod and at the container level. + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: demo +spec: + securityContext: + runAsUser: 1000 + runAsGroup: 3000 + fsGroup: 2000 + containers: + - name: api + image: registry.gitlab.com/web-hook/api:v1.0.39 + securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + runAsUser: 10000 + runAsNonRoot: true + seccompProfile: + type: RuntimeDefault + capabilities: + drop: + - ALL +``` + +## Network Policy +--- + +By default, Pods created in separate Namespaces can communicate with each other. To control Pod-to-Pod communications, Kubernetes has a NetworkPolicy resource. Based on Pod's labels, it can restrict ingress and egress communication for the selected Pods. + +The example below defines a NetworkPolicy restricting the database Pod to receive traffic from the backend Pod only. + +{{< image src="/images/learning-path/cka/security/netpol.png" width="100%" align="center" alt="" >}} + +The example below (from Kubernetes documentation) is more complex. It illustrates the full capabilities of NetworkPolicies. + +```yaml +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: test-network-policy + namespace: default +spec: + podSelector: + matchLabels: + role: db + policyTypes: + - Ingress + - Egress + ingress: + - from: + - ipBlock: + cidr: 172.17.0.0/16 + except: + - 172.17.1.0/24 + - namespaceSelector: + matchLabels: + project: myproject + - podSelector: + matchLabels: + role: frontend + ports: + - protocol: TCP + port: 6379 + egress: + - to: + - ipBlock: + cidr: 10.0.0.0/24 + ports: + - protocol: TCP + port: 5978 +``` + +It defines a NetworkPolicy for Pods with the label `role: db` managing incoming and outgoing traffic for those Pods. + +It authorizes **incoming traffic from** (logical OR): + +- Pods with IP addresses in a specific range +- Pods within a namespace +- Pods with specific labels in the current namespace + +It also authorizes **outgoing traffic to** (logical OR): + +- Pods with IP addresses in a specific range +- Pods within a specific namespace +- Pods with specific labels in the current namespace + +Both incoming and outgoing traffic are limited to specific ports. + +## Practice +--- + +You can now jump to the [Exercises part](./exercises/) to learn and practice the concepts above. + +{{< /chapterstyle >}} \ No newline at end of file diff --git a/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/storage.md b/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/storage.md new file mode 100644 index 000000000..a0b54f738 --- /dev/null +++ b/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/storage.md @@ -0,0 +1,158 @@ +--- +docType: "Chapter" +id: "storage" +chapterTitle: "Storage" +description: "Understand StorageClass, PV and PVCs stateful applications." +lectures: 10 +title: "Storage" +weight: 6 +--- + +{{< chapterstyle >}} + +This section is a refresher that provides an overview of the primary Kubernetes resources related to storage. At the end of this section, please complete the exercises to put these concepts into practice. + +## Volume +--- + +Volume is a property that can be defined in a Pod's specification (at `.spec.volumes`) and mounted in a container's filesystem (`.spec.containers.volumeMounts`). It allows decoupling a container from the storage solution and containers of the same pod to share data. + +Among the available types of volumes: + +- `emptyDir` +- `configMap` +- `Secret` +- `hostPath` +- `downwardAPI` + +The following specification defines a MongoDB Pod with an `emptyDir` volume mounted at `/data/db` in the container filesystem. It allows the data to persist outside the container's filesystem but still on the host filesystem. + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: mongo +spec: + containers: + - name: mongo + image: mongo:7.0 + volumeMounts: + - name: data + mountPath: /data/db + volumes: + - name: data + emptyDir: {} +``` + +**Note:** We should not use a volume for persistent storage; instead we should use PersistentVolume and PersistentVolumeClaim. + +## PersistentVolume +--- + +A PersistentVolume (PV) is a resource used to provide storage, either statically or dynamically. It decouples an application from the storage solution. + +Various types of PersistentVolume are available, including: + +- `cephfs` +- `csi` +- `fc` +- `hostPath` +- `iscsi` +- `local` +- `nfs` +- `rbd` + +The main properties of a PV are the following: + +### volumeMode + +Defines how the volume is presented to a Pod: + +- `Filesystem` (default) +- `Block` + +### accessModes + +Specifies how we can mount the volume in the Pod: + +- `ReadWriteOnce (RWO)`: the volume can be mounted as read-write by a single Node +- `ReadOnlyMany (ROX)`: the volume can be mounted as read-only by multiple Nodes at the same time +- `ReadWriteMany (RWX)`: the volume can be mounted as read-write by multiple Nodes at the same time +- `ReadWriteOncePod (RWOP)`: the volume can be mounted as read-write by a single Pod only + +### persistentVolumeReclaimPolicy + +Defines what happens to the PersistentVolume after the associated PersistentVolumeClaim is deleted: + +- `Retain` (default if manually created): the PersistentVolume is not deleted, a manual data recovery is possible +- `Delete` (default if dynamically created): the PersistentVolume and the underlying storage are deleted + +The specification below defines a PersistentVolume that will be tied to a single Node and offer 1G of storage from this Node's filesystem. + +```yaml +apiVersion: v1 +kind: PersistentVolume +metadata: + name: pv +spec: + accessModes: + - ReadWriteOnce + capacity: + storage: 1Gi + hostPath: + path: /tmp/data +``` + +## PersistentVolumeClaim +--- + +A PersistentVolumeClaim (PVC) is a storage request. It specifies the storage requirements regarding size and access mode, and is bound to a PersistentVolume that meets those requirements. + +The following specification defines a PersistentVolumeClaim, which requests 500 MB of storage with a RWO mode. + +```yaml +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: pvc +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 500Mi +``` + +A Pod can request storage by referencing the PVC in its specification as follows. When a PVC is bound to a PV, the PV is mounted into the container's filesystem. + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: db +spec: + containers: + - image: mongo:5.0 + name: mongo + volumeMounts: + - name: data + mountPath: /data/db + volumes: + - name: data + persistentVolumeClaim: + claimName: pvc +``` + +## StorageClass +--- + +A StorageClass defines how to dynamically create PersistentVolumes without needing to pre-provision them. + +More information in the official documentation at https://kubernetes.io/docs/concepts/storage/ + +## Practice +--- + +You can now jump to the [Exercises part](./exercises/) to learn and practice the concepts above. + +{{< /chapterstyle >}} \ No newline at end of file diff --git a/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/troubleshooting.md b/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/troubleshooting.md new file mode 100644 index 000000000..336a83f2e --- /dev/null +++ b/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/troubleshooting.md @@ -0,0 +1,470 @@ +--- +docType: "Chapter" +id: "troubleshooting" +chapterTitle: "Troubleshooting" +description: "Troubleshoot clusters components, nodes, network and applications." +lectures: 10 +title: "Troubleshooting" +weight: 8 +--- + +{{< chapterstyle >}} + +This section is a refresher that provides an overview of the main concepts used to troubleshoot a Kubernetes cluster. At the end of this section, please complete the exercises to put these concepts into practice. + +## Log management +--- + +Logs allow users to: +- follow the cluster's activity +- analyze errors + +We must decouple log management from the workload (Containers, Pods) and from the Nodes. + +Among the **best practices**: +- a container must log on stdout/stderr +- we should ship logs to a centralized logging solution + +### Different levels + +The following picture illustrates how we can configure logging on a cluster: + +**Node level**: the container runtime stores the logs on the Node's filesystem + +**Cluster level**: +- the container directly ships logs to a centralized logging system +- the container runtime stores the logs on the Node's filesystem, and then an external process (usually deployed as a DaemonSet) reads these logs and ships them to a centralized system +- a sidecar container is used to generate the logs on stdin/stdout, next the container runtime stores the logs on the Node's filesystem, and then an external process reads these logs and ships them to a centralized system + +{{< image src="/images/learning-path/cka/troubleshooting/logging-levels.png" width="100%" align="center" alt="" >}} + +### Pods & Containers logs + +The common way to get a Pod's logs is using kubectl. First, we run a Pod based on the ghost image. + +```bash +kubectl run ghost --image=ghost:4 +``` + +Next, we can query the Pod's logs. + +```bash +$ kubectl logs ghost +[2025-04-24 13:05:39] INFO Ghost is running in production... +[2025-04-24 13:05:39] INFO Your site is now available on http://localhost:2368/ +[2025-04-24 13:05:39] INFO Ctrl+C to shut down +[2025-04-24 13:05:39] INFO Ghost server started in 0.974s +... +``` + +We can also find these logs on the filesystem of the Node this Pod is running on. The following command tells us the Pod is running on worker1. + +```bash +kubectl get po -o wide +NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES +ghost 1/1 Running 0 6m32s 10.0.0.96 worker1 +``` + +The `/var/log/pods` folder on that Node contains the logs of all the Pods running on that Node, including the logs of the ghost Pod. + +```bash +$ sudo ls /var/log/pods +default_ghost_c502bf3e-7671-4488-af0c-5a2f0908db41 kube-system_cilium-envoy-c5vhw_6ac2a2af-3945-4069-b1cc-bb257ace3884 +default_mongo_331fa933-e9cf-42f8-94c0-93fe5e5d6e82 kube-system_cilium-vhtbk_b9548366-e92e-4c97-ba78-87c7b702cf28 +default_podinfo_c146441a-ba30-41cd-8e99-80feb7f12afe kube-system_kube-proxy-szfrd_17d684c2-a4ff-4316-b855-6f1ff63e5a0b +``` + +We get the same content we had using kubectl. + +```bash +$ sudo cat /var/log/pods/default_ghost_c502bf3e-7671-4488-af0c-5a2f0908db41/ghost/0.log +2025-04-24T13:05:39.159890159Z stdout F [2025-04-24 13:05:39] INFO Ghost is running in production... +2025-04-24T13:05:39.161339474Z stdout F [2025-04-24 13:05:39] INFO Your site is now available on http://localhost:2368/ +2025-04-24T13:05:39.161693705Z stdout F [2025-04-24 13:05:39] INFO Ctrl+C to shut down +2025-04-24T13:05:39.165232139Z stdout F [2025-04-24 13:05:39] INFO Ghost server started in 0.974s +``` + +Still from worker1, we can get containers' logs in `/var/log/containers` + +```bash +$ sudo ls /var/log/containers +cilium-envoy-c5vhw_kube-system_cilium-envoy-e73d79abd769cfa05f392807ca4ebacf7103b0049c19b2787e0e5128afe42f4d.log +cilium-vhtbk_kube-system_apply-sysctl-overwrites-7ac45e4b75302714bdbcc44da611b733a40a199a19d160e01a9496749920f043.log +cilium-vhtbk_kube-system_cilium-agent-9226fca5f50ebdc539e8045cece3e3cc9606b82e66ae67fb87a35b270fb71b96.log +cilium-vhtbk_kube-system_clean-cilium-state-5d9e303fbc85ad62e0f2e41be4b74ecfe6e7d989160dcdcef915006f5b6b308d.log +cilium-vhtbk_kube-system_config-3b3d66e2580dcf07baefe4dd9c792a08f7a41e16789ff07d6cb737b461b6b1a2.log +cilium-vhtbk_kube-system_install-cni-binaries-c039337274f5f8e09f0d886e2be9ae71356dff5cf25a2c0ed153b3d3bf2fe656.log +cilium-vhtbk_kube-system_mount-bpf-fs-935a17f160e800340dd1b9a7bdc294be3eec8628208fd0c36bb924b6345e9ed4.log +cilium-vhtbk_kube-system_mount-cgroup-b20b1292b4a4daed58ec5b7291b1a3744b249b19eca208ede650761b85a2f7fa.log +ghost_default_ghost-f0154dc4e4572d3827ba70717fba1caf19e0dcbea9060cde65965a424f9f3a3e.log <- this one +kube-proxy-szfrd_kube-system_kube-proxy-d2301ac47955299ea54ed4ed53a19d3af51b1f52156f01271a15a417db5fdd8c.log +mongo_default_mongo-70628c097a2032abd76d0716e62635378befb7efb077b787581eb86d195535f4.log +podinfo_default_podinfo-5e99fed167e8338e2d11b8e337fb3490522dc7c601ee83e60f85e80c5d7d4f4a.log +``` + +### Control plane logs + +We can get the logs of the control plane components (API Server, etcd, controller-manager, and scheduler) with kubectl. The following command allows us to get the logs of the API Server running on the controlplane Node. + +```bash +kubectl -n kube-system logs kube-apiserver-controlplane +``` + +The logs of these components are available on the controlplane Node, which is the Node they are running on. + +```bash +$ sudo ls -al /var/log/pods +total 36 +drwxr-xr-x 9 root root 4096 Apr 22 15:10 kube-system_cilium-cm954_1af09dcb-9738-458f-bb94-335505f7d713 +drwxr-xr-x 3 root root 4096 Apr 22 15:10 kube-system_cilium-envoy-ksqxr_d3a09feb-0827-4e80-84e6-1a960377bf0c +drwxr-xr-x 3 root root 4096 Apr 22 15:05 kube-system_etcd-controlplane_05261863f509698b43b78850b9ccfe8f +drwxr-xr-x 3 root root 4096 Apr 22 15:05 kube-system_kube-apiserver-controlplane_80c4d1003f6284601e0aa670932b5ee7 +drwxr-xr-x 3 root root 4096 Apr 22 15:05 kube-system_kube-controller-manager-controlplane_2c3d35add706c540cb5a3ad3a246bee9 +drwxr-xr-x 3 root root 4096 Apr 22 15:05 kube-system_kube-proxy-n5lct_6b0c2017-b4b1-4ef3-9678-e3d3dc8687e8 +drwxr-xr-x 3 root root 4096 Apr 22 15:05 kube-system_kube-scheduler-controlplane_4a834c796528f3fc43f3dadb50f3bd73 +... +``` + +### Kubelet logs + +The kubelet agent, running on each Node of the cluster, is managed by systemd. We can use journalctl to get its logs. + +```bash +sudo journalctl -u kubelet | less +``` + +## Metrics management +--- + +In Kubernetes, metrics come in various types, originate from different layers of the stack, and are exposed by multiple components. + +**Types of Metrics**: +- CPU / RAM usage +- Disk I/O +- Network activity +- Request and error rates + +**Sources of Metrics**: +- Cluster-wide +- Control plane +- Individual Nodes +- Pods and containers +- Applications + +**Metrics Producers**: +- cAdvisor (embedded in kubelet) +- Metrics Server +- Kubernetes API Server +- Node Exporter +- kube-state-metrics + +### Prometheus-based solution + +The Prometheus stack is a widely used solution to manage Metrics in a Kubernetes cluster. + +{{< image src="/images/learning-path/cka/troubleshooting/monitoring-prometheus.png" width="100%" align="center" alt="" >}} + +### Metrics server + +The metrics-server is a lightweight component that is not installed by default in Kubernetes. It gets CPU / RAM usage in real time but does not store history. Other resources, such as HorizontalPodAutoscaler (HPA), use it to increase/decrease the number of Pods based on resource consumption. + +The metrics-server brings additional kubectl commands to get the usage of resources in the cluster. + +- **Getting the CPU and RAM in use by the cluster's Nodes** + +```bash +kubectl top nodes +``` + +- **Getting the CPU and RAM in use by individual Pods** + +```bash +kubectl top pods +``` + +## Cluster components +--- + +Each control plane component is a static Pod. The `/etc/kubernetes/manifests` folder of the controlplane Node contains all their YAML specifications. These Pods are directly managed by kubelet. + +```bash +$ ls /etc/kubernetes/manifests +etcd.yaml kube-apiserver.yaml kube-controller-manager.yaml kube-scheduler.yaml +``` + +For each static Pod, kubelet automatically creates a mirror Pod that appears in the Kubernetes API. + +```bash +$ kubectl get po -n kube-system +coredns-64897985d-gfslg 1/1 Running 0 46h +coredns-64897985d-q7qd2 1/1 Running 0 107s +etcd-controlplane 1/1 Running 0 5d17h <- mirror Pod +kube-apiserver-controlplane 1/1 Running 0 5d17h <- mirror Pod +kube-controller-manager-controlplane 1/1 Running 1 (3d1h ago) 5d17h <- mirror Pod +kube-proxy-25w94 1/1 Running 0 5d17h +kube-proxy-778cb 1/1 Running 0 5d17h +kube-proxy-h4hbh 1/1 Running 0 5d17h +kube-scheduler-controlplane 1/1 Running 1 (3d1h ago) 5d17h <- mirror Pod +weave-net-66dtm 2/2 Running 1 (5d17h ago) 5d17h +weave-net-pfcrp 2/2 Running 1 (5d17h ago) 5d17h +weave-net-zxchk 2/2 Running 1 (5d17h ago) 5d17h +``` + +## Troubleshooting - Examples +--- + +### Application failure - Example 1 + +The following specification seems valid for deploying a Pod based on the elasticsearch image. + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: es +spec: + containers: + - image: elasticsearch:7.6.2 + name: es +``` + +After waiting a few dozen seconds following the creation of this Pod, we begin to see some errors. + +```bash +$ kubectl get pods/es -w +NAME READY STATUS RESTARTS AGE +es 0/1 ContainerCreating 0 10s +es 1/1 Running 0 22s +es 0/1 Error 1 58s +es 0/1 CrashLoopBackOff 1 70s +``` + +To understand the origin of these errors, we first need to use the describe command to get more details. + +```bash +$ kubectl describe po/es +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + … + Normal Pulled 52s (x4 over 3m16s) kubelet, workers-1i2u Container image "elasticsearch:7.6.2" already present on machine + Warning BackOff 19s (x9 over 2m57s) kubelet, workers-1i2u Back-off restarting failed container +``` + +Next, we verify the application logs. It shows the root cause: the value of the kernel property `vm.max_map_count` is too low. + +```bash +$ kubectl logs po/es +[2020-03-15T17:42:15,417][INFO ][o.e.b.BootstrapChecks ] [hK4xzxV] bound or publishing to a non-loopback address, enforcing bootstrap checks +ERROR: [1] bootstrap checks failed +[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] +[2020-03-15T17:42:15,429][INFO ][o.e.n.Node ] [hK4xzxV] stopping ... +[2020-03-15T17:42:15,483][INFO ][o.e.n.Node ] [hK4xzxV] stopped +``` + +We should use an initContainer and an env var to fix the thing for this specific example. + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: es +spec: + initContainers: + - name: increase-vm-max-map + image: busybox + command: ["sysctl", "-w", "vm.max_map_count=262144"] + securityContext: + privileged: true + containers: + - image: elasticsearch:7.6.2 + name: es + env: + - name: discovery.type + value: single-node +``` + +### Application failure - Example 2 + +Let's consider a Pod exposed with a Service. + +```bash +kubectl get po,svc -l app=ghost +NAME READY STATUS RESTARTS AGE +pod/ghost 1/1 Running 0 88s + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/ghost NodePort 10.43.121.142 2368:30526/TCP 88s +``` + +It happens that the application is not reachable via the NodePort provided. + +{{< image src="/images/learning-path/cka/troubleshooting/application-failure-2.png" width="100%" align="center" alt="" >}} + +In this case, we can first describe the Service and check the Endpoints. In the following example, **Endpoints is empty**, which indicates we did not configure the Service correctly. + +```bash +k describe svc ghost +Name: ghost +Namespace: default +Labels: app=ghost +Annotations: +Selector: run=ghost +Type: NodePort +IP Family Policy: SingleStack +IP Families: IPv4 +IP: 10.43.121.142 +IPs: 10.43.121.142 +Port: 2368/TCP +TargetPort: 2368/TCP +NodePort: 30526/TCP +Endpoints: +Session Affinity: None +External Traffic Policy: Cluster +Events: +``` + +The list of Endpoints is empty, so the service does not expose a single Pod. + +There is a mismatch between the Service's selector and the pod labels: +- Service's selector is `run: ghost` +- Pod's label is `app: ghost` + +In this case, we need to change one of them to ensure they match. + +### Failure of the API Server + +If kubectl commands hang, that may be because the API Server is not available. + +```bash +$ kubectl get po +... hanging +``` + +From the controlplane, we first check the kubelet's logs. In this example, the logs indicate the API Server encounters a problem to start. + +```bash +Checking kubelet logs +$ sudo journalctl -u kubelet | less +sudoMar 31 09:49:17 controlplane kubelet[72558]: E0331 09:49:17.443398 72558 pod_workers.go:919] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-apiserver\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=kube-apiserver pod=kube-apiserver-controlplane_kube-system(1379f6cdef52f9b598e745122eb20d6f)\"" pod="kube-system/kube-apiserver-controlplane" podUID=1379f6cdef52f9b598e745122eb20d6f +Mar 31 09:49:18 controlplane kubelet[72558]: E0331 09:49:18.426742 72558 kubelet_node_status.go:460] "Error updating node status, will retry" err="error getting node \"controlplane\": Get \"https://194.182.171.68:6443/api/v1/nodes/controlplane?timeout=10s\": context deadline exceeded" +… +``` + +We retrieve the name of the API Server's log file from the `/var/log/pods` folder on the controlplane Node. + +```bash +$ ls -al /var/log/pods +total 40 +drwxr-xr-x 10 root root 4096 Mar 31 09:45 . +drwxrwxr-x 10 root syslog 4096 Mar 31 00:00 .. +drwxr-xr-x 3 root root 4096 Mar 29 10:38 kube-system_coredns-64897985d-gfslg_adaa9cfe-42a4-4bc7-b5aa-eb0313b59fe7 +drwxr-xr-x 3 root root 4096 Mar 28 08:55 kube-system_coredns-64897985d-mvp4t_bcfea69a-d6cc-4baf-a795-acad8fab2e47 +drwxr-xr-x 3 root root 4096 Mar 25 16:05 kube-system_etcd-controlplane_6d694021cab77267a88779a2268199e6 +drwxr-xr-x 3 root root 4096 Mar 31 09:44 kube-system_kube-apiserver-controlplane_1379f6cdef52f9b598e745122eb20d6f <- this one +drwxr-xr-x 3 root root 4096 Mar 26 13:46 kube-system_kube-controller-manager-controlplane_94d947d1226129a82876a3b7d829bbfc +drwxr-xr-x 3 root root 4096 Mar 25 16:06 kube-system_kube-proxy-25w94_0c17e655-c491-43f6-b012-0eab0c7f8071 +drwxr-xr-x 3 root root 4096 Mar 26 13:46 kube-system_kube-scheduler-controlplane_415ed7d85341035184628df29257fa2f +drwxr-xr-x 5 root root 4096 Mar 25 16:06 kube-system_weave-net-66dtm_cef2efd7-9ea6-4604-a871-53ab915a7a84 +``` + +From that file, we directly understand why the API Server cannot start: an invalid configuration option is used in its specification. + +```bash +sudo cat kube-system_kube-apiserver-controlplae_1379f6cdef52f9b598e745122eb20d6f/kube-apiserver/8.log +2022-03-31T10:00:27.785052657Z stderr F I0331 10:00:27.784813 1 server.go:565] external host was not specified, using 10.62.50.215 +2022-03-31T10:00:27.785838518Z stderr F E0331 10:00:27.785689 1 run.go:74] "command failed" err="enable-admission-plugins plugin \"WRONG_STUFF_HERE\" is unknown" +``` + +Still, from the controlplane Node, we can check the API Server specification (`/etc/kubernetes/manifests/kube-apiserver.yaml`) and fix the incorrect configuration. + +```yaml +apiVersion: v1 +kind: Pod +metadata: + annotations: + kubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint: 10.62.50.215:6443 + creationTimestamp: null + labels: + component: kube-apiserver + tier: control-plane + name: kube-apiserver + namespace: kube-system +spec: + containers: + - command: + - kube-apiserver + - --advertise-address=10.62.50.215 + - --allow-privileged=true + - --authorization-mode=Node,RBAC + - --client-ca-file=/etc/kubernetes/pki/ca.crt + - --enable-admission-plugins=NodeRestriction,WRONG_STUFF_HERE <- ,WRONG_STUFF_HERE needs to be removed + - --enable-bootstrap-token-auth=true + ... +``` + +Once the specification is changed, kubelet automatically restarts the API Server Pod. + +### Failure of a worker node + +Sometimes, a Node may not be in the Ready state as illustrated below. + +```bash +$ kubectl get nodes +NAME STATUS ROLES AGE VERSION +controlplane Ready control-plane 5d19h v1.32.2 +worker1 NotReady 5d19h v1.32.2 <- This Node does not seem to work +worker2 Ready 5d19h v1.32.2 +``` + +We start by getting more information about this Node to troubleshoot this issue. + +```bash +kubectl describe nodes worker1 +Name: worker1 +Taints: node.kubernetes.io/unreachable:NoExecute + node.kubernetes.io/unreachable:NoSchedule +… +Conditions: + Type Status LastHeartbeatTime LastTransitionTime Reason Message + ---- ------ ----------------- ------------------ ------ ------- + NetworkUnavailable False Fri, 25 Mar 2022 16:06:39 +0000 Fri, 25 Mar 2022 16:06:39 +0000 WeaveIsUp Weave pod has set this + MemoryPressure Unknown Thu, 31 Mar 2022 11:40:16 +0000 Thu, 31 Mar 2022 11:43:35 +0000 NodeStatusUnknown Kubelet stopped posting node status. + DiskPressure Unknown Thu, 31 Mar 2022 11:40:16 +0000 Thu, 31 Mar 2022 11:43:35 +0000 NodeStatusUnknown Kubelet stopped posting node status. + PIDPressure Unknown Thu, 31 Mar 2022 11:40:16 +0000 Thu, 31 Mar 2022 11:43:35 +0000 NodeStatusUnknown Kubelet stopped posting node status. + Ready Unknown Thu, 31 Mar 2022 11:40:16 +0000 Thu, 31 Mar 2022 11:43:35 +0000 NodeStatusUnknown Kubelet stopped posting node status. +… +``` + +The result above indicates that the kubelet process running on worker1 has stopped posting the Node's status, which might indicate that process no longer runs. In that case, we can check the status of the kubelet systemd service on worker1. + +```bash +Checking the status of the node's kubelet +sudo systemctl status kubelet +● kubelet.service - kubelet: The Kubernetes Node Agent + Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled) + Drop-In: /etc/systemd/system/kubelet.service.d + └─10-kubeadm.conf + Active: inactive (dead) since Thu 2022-03-31 11:42:53 UTC; 4min 29s ago + Docs: https://kubernetes.io/docs/home/ + Process: 66511 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=0/SUCCESS) + Main PID: 66511 (code=exited, status=0/SUCCESS) + +Mar 31 11:35:14 worker1 kubelet[66511]: I0331 11:35:14.894783 66511 image_gc_manager.go:327] "Attempting to delete unused images" +Mar 31 11:35:14 worker1 kubelet[66511]: I0331 11:35:14.916929 66511 eviction_manager.go:349] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" +Mar 31 11:35:14 worker1 kubelet[66511]: I0331 11:35:14.916992 66511 eviction_manager.go:367] "Eviction manager: pods ranked for eviction" pods=[kube-system/weave-net-zxchk kube-system/kube-proxy-778cb] +... +``` + +As kubelet is not running, we can restart it. + +```bash +sudo systemctl restart kubelet +``` + +## Practice +--- + +You can now jump to the [Exercises part](./exercises/) to learn and practice the concepts above. + +{{< /chapterstyle >}} \ No newline at end of file diff --git a/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/workload.md b/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/workload.md new file mode 100644 index 000000000..9768c0330 --- /dev/null +++ b/content/en/learning-paths/mastering-kubernetes-for-engineers/cka/content/workload.md @@ -0,0 +1,365 @@ +--- +docType: "Chapter" +id: "workload" +chapterTitle: "Workloads" +description: "Create and manage Pods, Deployments, and other workload resources." +lectures: 10 +title: "Workloads" +weight: 3 +--- + +{{< chapterstyle >}} + +Create and manage Pods, Deployments, and other workload resources. + +This section is a refresher that provides an overview of the primary Kubernetes resources related to workloads. At the end of this section, please complete the exercises to put these concepts into practice. + +{{< image src="/images/learning-path/cka/workload/main-resources.png" width="100%" align="center" alt="" >}} + +## Pod +--- + +### Purpose + +A Pod is the smallest workload unit in Kubernetes. It's an abstraction containing one or more containers in charge of running applications. + +### Sample specification + +Below is the simplest version of a Pod, this one runs a container based on the **stefanprodan/podinfo** image. + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: podinfo +spec: + containers: + - name: podinfo + image: stefanprodan/podinfo +``` + +We create the Pod with the usual kubectl command from this YAML definition. + +```bash +kubectl apply -f pod.yaml +``` + +As a Pod does not expose the application it is running (this is the role of the Service resource, which we'll detail later in this workshop), we can access the application using a port-forward command as follows. + +```bash +kubectl port-forward podinfo 9898:9898 --address 0.0.0.0 +``` + +This command opens port 9898 on the machine it is run from and forwards traffic to port 9898 in the Pod. The `–address 0.0.0.0` flag ensures this port is available on all the network interfaces of the host machine (otherwise limited to localhost). + +{{< image src="/images/learning-path/cka/workload/podinfo.png" width="100%" align="center" alt="" >}} + +### Enhanced specification + +The simple specification we saw above is too simple and must not be run in a production environment. Below is a more complete specification, including additional properties: + +- **resources**, which specifies the requirements and limits in terms of CPU and RAM +- **livenessProbe**, which ensures the application is healthy +- **readinessProbe**, which ensures the application is ready to accept requests +- **securityContext**, which adds security constraints + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: podinfo + labels: + app: podinfo +spec: + containers: + - image: stefanprodan/podinfo:6.1.0 + name: podinfo + resources: + requests: + cpu: 50m + memory: 64Mi + limits: + cpu: 50m + memory: 64Mi + livenessProbe: + httpGet: + path: /healthz + port: 9898 + initialDelaySeconds: 3 + periodSeconds: 3 + readinessProbe: + httpGet: + path: /readyz + port: 9898 + initialDelaySeconds: 3 + periodSeconds: 3 + securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + runAsUser: 10000 + runAsNonRoot: true + seccompProfile: + type: RuntimeDefault + capabilities: + drop: + - ALL +``` + +## Deployment +--- + +{{< image src="/images/learning-path/cka/workload/deployment.png" width="100%" align="center" alt="" >}} + +### Purpose + +A Deployment runs a given number of identical Pods across the cluster. The number of replicas can easily be scaled (manually or with an HPA). + +### Sample specification + +The following specification defines a Deployment in charge of 5 Pods based on the **nginx:1.24** image. + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: www +spec: + replicas: 5 + selector: + matchLabels: + app: www + template: + metadata: + labels: + app: www + spec: + containers: + - name: www + image: nginx:1.24 + ports: + - containerPort: 80 +``` + +## DaemonSet +--- + +{{< image src="/images/learning-path/cka/workload/daemonset.png" width="100%" align="center" alt="" >}} + +### Purpose + +A DaemonSet ensures a Pod (usually an agent) is running on each cluster's Node. + +### Sample specification + +The following specification defines a DaemonSet in charge of running a fluentbit Pod on each node of the cluster. + +```yaml +apiVersion: apps/v1 +kind: DaemonSet +metadata: + name: fluent-bit +spec: + selector: + matchLabels: + k8s-app: fluent-bit-logging + template: + metadata: + labels: + k8s-app: fluent-bit-logging + spec: + containers: + - name: fluent-bit + image: fluent/fluent-bit:1.5 + volumeMounts: + - name: varlog + mountPath: /var/log + - name: varlogcontainers + mountPath: /var/log/containers + volumes: + - name: varlog + hostPath: + path: /var/log + - name: varlogcontainers + hostPath: + path: /var/log/containers +``` + +## Job +--- + +### Purpose + +A Job allows running several Pods in parallel or sequence. This must not be used to run long-running tasks, like application servers. + +### Sample specification + +The following specification defines a Job running 3 Pods in parallel to train a machine learning model. + +```yaml +apiVersion: batch/v1 +kind: Job +metadata: + name: training +spec: + completions: 3 + parallelism: 3 + template: + spec: + restartPolicy: OnFailure + containers: + - name: training + image: org/ml-training:1.2 +``` + +## CronJob +--- + +### Purpose + +A CronJob is in charge of launching Jobs according to a schedule, similar to the Linux crontab. + +### Sample specification + +The following specification defines a CronJob in charge of running a backup Job every 6 hours. + +```yaml +apiVersion: batch/v1 +kind: CronJob +metadata: + name: dump +spec: + schedule: "0 */6 * * *" + jobTemplate: + spec: + template: + spec: + restartPolicy: OnFailure + containers: + - name: dump + image: org/db-dump:2.3 +``` + +## ConfigMap +--- + +### Purpose + +A ConfigMap is not used to run Pods, unlike the resources above. Instead, we use it to configure the application running in Pods. A ConfigMap contains files or key/value pairs that we provide to the containers either: + +- via environment variables +- or mounted as a volume + +### Sample specification + +The following specification defines a ConfigMap containing a nginx configuration file. + +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: proxy-config +data: + nginx.conf: | + user nginx; + worker_processes 4; + pid /run/nginx.pid; + events { + worker_connections 768; + } + http { + server { + listen *:80; + location = /whoami { + proxy_pass http://whoami/; + } + } + } +``` + +### Usage in a Pod + +The following specification illustrates how to use this ConfigMap in a Pod. The **nginx.conf** file is made available in the `/etc/nginx/config.conf` in the Pod's container. + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: www +spec: + containers: + - name: proxy + image: nginx:1.24 + volumeMounts: + - name: config + mountPath: "/etc/nginx/" + volumes: + - name: config + configMap: + name: nginx-config +``` + +## Secret +--- + +### Purpose + +A Secret is very similar to a ConfigMap except that it is used to handle sensitive information (credentials, ssh keys) as it can be encrypted. As for ConfigMap, a Secret contains files or key/value pairs that we can provide to the containers either: + +- via environment variables +- or mounted as a volume + +When viewing a Secret's specification, we can see base64 encoded (not encrypted) content. + +### Sample specification + +The following specification defines a Secret containing an encoded MongoDB connection string. + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: mongo-credentials +data: + mongo_url: dG9rZW49Y2IzNDU2YTU0RUI1Cg== +type: Opaque +``` + +### Usage in a Pod + +The specification below illustrates how to use a Secret in a Pod. The Secret's unique key is available as an environment variable in the Pod's container. + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: api +spec: + containers: + - name: api + image: api:1.2 + env: + - name: MONGO_URL + valueFrom: + secretKeyRef: + name: mongo-credentials + key: mongo_url +``` + +## Namespaces +--- + +Namespaces are Kubernetes resources that allow grouping resources. The image below illustrates the Namespaces created by default. + +{{< image src="/images/learning-path/cka/workload/namespace-1.png" width="100%" align="center" alt="" >}} + +As Namespaces do not offer strong isolation, specific resources must be applied to a Namespace to limit CPU, RAM usage, and network rules allowed within and across Namespaces. Those resources are: + +- **ResourceQuota** +- **Limits** +- **NetworkPolicy** + +{{< image src="/images/learning-path/cka/workload/namespace-2.png" width="100%" align="center" alt="" >}} + +You can now jump to the [Exercises part](./exercises/) to learn and practice the concepts above. + +{{< /chapterstyle >}} \ No newline at end of file diff --git a/static/images/learning-path/cka/certifications/cka.png b/static/images/learning-path/cka/certifications/cka.png new file mode 100644 index 000000000..04f15bb51 Binary files /dev/null and b/static/images/learning-path/cka/certifications/cka.png differ diff --git a/static/images/learning-path/cka/certifications/ckad.png b/static/images/learning-path/cka/certifications/ckad.png new file mode 100644 index 000000000..532bbde6b Binary files /dev/null and b/static/images/learning-path/cka/certifications/ckad.png differ diff --git a/static/images/learning-path/cka/certifications/cks.png b/static/images/learning-path/cka/certifications/cks.png new file mode 100644 index 000000000..3c58f9252 Binary files /dev/null and b/static/images/learning-path/cka/certifications/cks.png differ diff --git a/static/images/learning-path/cka/certifications/kcna.png b/static/images/learning-path/cka/certifications/kcna.png new file mode 100644 index 000000000..eb2950b6f Binary files /dev/null and b/static/images/learning-path/cka/certifications/kcna.png differ diff --git a/static/images/learning-path/cka/certifications/kcsa.png b/static/images/learning-path/cka/certifications/kcsa.png new file mode 100644 index 000000000..9085f2e3f Binary files /dev/null and b/static/images/learning-path/cka/certifications/kcsa.png differ diff --git a/static/images/learning-path/cka/certifications/psi-browser.png b/static/images/learning-path/cka/certifications/psi-browser.png new file mode 100644 index 000000000..a15450b4f Binary files /dev/null and b/static/images/learning-path/cka/certifications/psi-browser.png differ diff --git a/static/images/learning-path/cka/creation/objectives.png b/static/images/learning-path/cka/creation/objectives.png new file mode 100644 index 000000000..df1cd8a0d Binary files /dev/null and b/static/images/learning-path/cka/creation/objectives.png differ diff --git a/static/images/learning-path/cka/creation/step-1.png b/static/images/learning-path/cka/creation/step-1.png new file mode 100644 index 000000000..99f8331f0 Binary files /dev/null and b/static/images/learning-path/cka/creation/step-1.png differ diff --git a/static/images/learning-path/cka/creation/step-2.png b/static/images/learning-path/cka/creation/step-2.png new file mode 100644 index 000000000..16861246b Binary files /dev/null and b/static/images/learning-path/cka/creation/step-2.png differ diff --git a/static/images/learning-path/cka/creation/step-3.png b/static/images/learning-path/cka/creation/step-3.png new file mode 100644 index 000000000..245add556 Binary files /dev/null and b/static/images/learning-path/cka/creation/step-3.png differ diff --git a/static/images/learning-path/cka/networking/ClusterIP.png b/static/images/learning-path/cka/networking/ClusterIP.png new file mode 100644 index 000000000..df94aa62c Binary files /dev/null and b/static/images/learning-path/cka/networking/ClusterIP.png differ diff --git a/static/images/learning-path/cka/networking/LoadBalancer.png b/static/images/learning-path/cka/networking/LoadBalancer.png new file mode 100644 index 000000000..9108dfa38 Binary files /dev/null and b/static/images/learning-path/cka/networking/LoadBalancer.png differ diff --git a/static/images/learning-path/cka/networking/NodePort.png b/static/images/learning-path/cka/networking/NodePort.png new file mode 100644 index 000000000..8d4407b43 Binary files /dev/null and b/static/images/learning-path/cka/networking/NodePort.png differ diff --git a/static/images/learning-path/cka/networking/cni.png b/static/images/learning-path/cka/networking/cni.png new file mode 100644 index 000000000..efb1c8406 Binary files /dev/null and b/static/images/learning-path/cka/networking/cni.png differ diff --git a/static/images/learning-path/cka/networking/container-to-container.png b/static/images/learning-path/cka/networking/container-to-container.png new file mode 100644 index 000000000..cbc701819 Binary files /dev/null and b/static/images/learning-path/cka/networking/container-to-container.png differ diff --git a/static/images/learning-path/cka/networking/encapsulated.png b/static/images/learning-path/cka/networking/encapsulated.png new file mode 100644 index 000000000..fabc7353a Binary files /dev/null and b/static/images/learning-path/cka/networking/encapsulated.png differ diff --git a/static/images/learning-path/cka/networking/ingress.png b/static/images/learning-path/cka/networking/ingress.png new file mode 100644 index 000000000..781150682 Binary files /dev/null and b/static/images/learning-path/cka/networking/ingress.png differ diff --git a/static/images/learning-path/cka/networking/pod-to-pod-different-nodes.png b/static/images/learning-path/cka/networking/pod-to-pod-different-nodes.png new file mode 100644 index 000000000..b91c3fcb5 Binary files /dev/null and b/static/images/learning-path/cka/networking/pod-to-pod-different-nodes.png differ diff --git a/static/images/learning-path/cka/networking/pod-to-pod-same-node.png b/static/images/learning-path/cka/networking/pod-to-pod-same-node.png new file mode 100644 index 000000000..59451cbcb Binary files /dev/null and b/static/images/learning-path/cka/networking/pod-to-pod-same-node.png differ diff --git a/static/images/learning-path/cka/networking/uncapsulated.png b/static/images/learning-path/cka/networking/uncapsulated.png new file mode 100644 index 000000000..9f3700a17 Binary files /dev/null and b/static/images/learning-path/cka/networking/uncapsulated.png differ diff --git a/static/images/learning-path/cka/scheduling/nodeName.png b/static/images/learning-path/cka/scheduling/nodeName.png new file mode 100644 index 000000000..5be431652 Binary files /dev/null and b/static/images/learning-path/cka/scheduling/nodeName.png differ diff --git a/static/images/learning-path/cka/scheduling/nodeSelector.png b/static/images/learning-path/cka/scheduling/nodeSelector.png new file mode 100644 index 000000000..9ffc5a1e4 Binary files /dev/null and b/static/images/learning-path/cka/scheduling/nodeSelector.png differ diff --git a/static/images/learning-path/cka/scheduling/scheduling.png b/static/images/learning-path/cka/scheduling/scheduling.png new file mode 100644 index 000000000..bcb470195 Binary files /dev/null and b/static/images/learning-path/cka/scheduling/scheduling.png differ diff --git a/static/images/learning-path/cka/scheduling/topologySpreadConstraint.png b/static/images/learning-path/cka/scheduling/topologySpreadConstraint.png new file mode 100644 index 000000000..1e8468661 Binary files /dev/null and b/static/images/learning-path/cka/scheduling/topologySpreadConstraint.png differ diff --git a/static/images/learning-path/cka/security/certificate.png b/static/images/learning-path/cka/security/certificate.png new file mode 100644 index 000000000..98e1b6214 Binary files /dev/null and b/static/images/learning-path/cka/security/certificate.png differ diff --git a/static/images/learning-path/cka/security/control-plane.png b/static/images/learning-path/cka/security/control-plane.png new file mode 100644 index 000000000..93f06b11e Binary files /dev/null and b/static/images/learning-path/cka/security/control-plane.png differ diff --git a/static/images/learning-path/cka/security/jwt.png b/static/images/learning-path/cka/security/jwt.png new file mode 100644 index 000000000..d45aa5af9 Binary files /dev/null and b/static/images/learning-path/cka/security/jwt.png differ diff --git a/static/images/learning-path/cka/security/netpol.png b/static/images/learning-path/cka/security/netpol.png new file mode 100644 index 000000000..aa36d29c0 Binary files /dev/null and b/static/images/learning-path/cka/security/netpol.png differ diff --git a/static/images/learning-path/cka/security/rbac-resources.png b/static/images/learning-path/cka/security/rbac-resources.png new file mode 100644 index 000000000..f2c2ebc3f Binary files /dev/null and b/static/images/learning-path/cka/security/rbac-resources.png differ diff --git a/static/images/learning-path/cka/security/rbac.png b/static/images/learning-path/cka/security/rbac.png new file mode 100644 index 000000000..da0ce971d Binary files /dev/null and b/static/images/learning-path/cka/security/rbac.png differ diff --git a/static/images/learning-path/cka/troubleshooting/application-failure-2.png b/static/images/learning-path/cka/troubleshooting/application-failure-2.png new file mode 100644 index 000000000..271d2dd37 Binary files /dev/null and b/static/images/learning-path/cka/troubleshooting/application-failure-2.png differ diff --git a/static/images/learning-path/cka/troubleshooting/logging-levels.png b/static/images/learning-path/cka/troubleshooting/logging-levels.png new file mode 100644 index 000000000..94905d1f7 Binary files /dev/null and b/static/images/learning-path/cka/troubleshooting/logging-levels.png differ diff --git a/static/images/learning-path/cka/troubleshooting/monitoring-prometheus.png b/static/images/learning-path/cka/troubleshooting/monitoring-prometheus.png new file mode 100644 index 000000000..ebeecd410 Binary files /dev/null and b/static/images/learning-path/cka/troubleshooting/monitoring-prometheus.png differ diff --git a/static/images/learning-path/cka/workload/daemonset.png b/static/images/learning-path/cka/workload/daemonset.png new file mode 100644 index 000000000..0f9e55154 Binary files /dev/null and b/static/images/learning-path/cka/workload/daemonset.png differ diff --git a/static/images/learning-path/cka/workload/deployment.png b/static/images/learning-path/cka/workload/deployment.png new file mode 100644 index 000000000..b30cc0595 Binary files /dev/null and b/static/images/learning-path/cka/workload/deployment.png differ diff --git a/static/images/learning-path/cka/workload/main-resources.png b/static/images/learning-path/cka/workload/main-resources.png new file mode 100644 index 000000000..df9d62780 Binary files /dev/null and b/static/images/learning-path/cka/workload/main-resources.png differ diff --git a/static/images/learning-path/cka/workload/namespace-1.png b/static/images/learning-path/cka/workload/namespace-1.png new file mode 100644 index 000000000..496e1bdda Binary files /dev/null and b/static/images/learning-path/cka/workload/namespace-1.png differ diff --git a/static/images/learning-path/cka/workload/namespace-2.png b/static/images/learning-path/cka/workload/namespace-2.png new file mode 100644 index 000000000..c440aefbd Binary files /dev/null and b/static/images/learning-path/cka/workload/namespace-2.png differ diff --git a/static/images/learning-path/cka/workload/podinfo.png b/static/images/learning-path/cka/workload/podinfo.png new file mode 100644 index 000000000..6dbba73df Binary files /dev/null and b/static/images/learning-path/cka/workload/podinfo.png differ