Kubernetes: Orchestrating Containers at Scale¶

What this lesson gives you

Kubernetes is the production operating system for containers. It runs your application across many nodes, handles failures automatically, scales based on load, and delivers zero-downtime deployments. The declarative model — desired state vs. current state — is the central concept to internalize.

Estimated time: 22 min read · Part: Docker & Kubernetes

Learning Objectives

After this lesson you will be able to: (1) explain the Kubernetes declarative model and the control loop; (2) define Pod, Deployment, Service, ConfigMap, and Secret; (3) read and write basic Kubernetes YAML manifests for a Python service; (4) describe liveness and readiness probes; (5) explain the HorizontalPodAutoscaler; (6) use the kubectl commands essential for debugging; (7) frame Kubernetes adoption conversations for enterprise clients.

The Declarative Model: Desired State¶

Kubernetes works on a desired state model. You declare what you want ("I want 3 replicas of this container, always running") in a YAML manifest and apply it to the cluster. Kubernetes continuously compares desired state to actual state and acts to reconcile any difference. A Pod crashes? Kubernetes restarts it. A node fails? Kubernetes schedules the Pod on another node. This control loop — observe actual state, compare to desired, act to close the gap — runs continuously for every object in the cluster.

Kubernetes as a Thermostat

A thermostat implements the same control loop: you declare a desired state (temperature = 20°C), it observes the actual state (currently 17°C), and takes action (turn on heating) to close the gap — then observes again. You never say "turn on heating" — you say "I want it to be 20°C." Kubernetes is the thermostat for your application fleet: declare "I want 3 replicas of payments-api running on port 8000" and Kubernetes makes it so, continuously, even as nodes fail and containers crash.

flowchart TB
    subgraph ControlPlane ["Control Plane"]
        API[API Server]
        ETCD[etcd<br/>desired state store]
        SCHED[Scheduler]
        CM[Controller<br/>Manager]
    end
    subgraph Worker1 ["Worker Node 1"]
        KL1[kubelet]
        P1[Pod: payments-api]
        P2[Pod: payments-api]
    end
    subgraph Worker2 ["Worker Node 2"]
        KL2[kubelet]
        P3[Pod: payments-api]
    end
    User([kubectl apply]) --> API
    API --> ETCD
    CM -->|reconcile loop| API
    SCHED --> KL1
    SCHED --> KL2
    KL1 --> P1
    KL1 --> P2
    KL2 --> P3

Core Objects: Pod, Deployment, Service¶

A Pod is the smallest deployable unit in Kubernetes — one or more containers that share a network namespace and storage. Pods are ephemeral: they can be killed and replaced at any time. You almost never create Pods directly. A Deployment manages a set of identical Pods, ensuring the desired replica count is always maintained and orchestrating rolling updates. A Service is a stable network endpoint that routes traffic to the current set of healthy Pods — it provides a consistent hostname and load balancing, decoupling clients from the ephemeral Pod IP addresses.

k8s/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payments-api
  labels:
    app: payments-api
spec:
  replicas: 3                  # desired number of Pods — Kubernetes maintains this
  selector:
    matchLabels:
      app: payments-api        # this Deployment manages Pods with this label
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0        # never kill a Pod before the replacement is ready
      maxSurge: 1              # allow 1 extra Pod during the update
  template:
    metadata:
      labels:
        app: payments-api
    spec:
      containers:
        - name: api
          image: ghcr.io/myorg/payments-api@sha256:a1b2c3...  # pin by digest!
          ports:
            - containerPort: 8000
          env:
            - name: DATABASE_URL
              valueFrom:
                secretKeyRef:
                  name: payments-secrets
                  key: database-url     # inject from Secret, not hardcoded

          # Readiness probe: don't send traffic until this passes
          readinessProbe:
            httpGet:
              path: /health
              port: 8000
            initialDelaySeconds: 5
            periodSeconds: 10

          # Liveness probe: restart container if this fails 3 times
          livenessProbe:
            httpGet:
              path: /health
              port: 8000
            initialDelaySeconds: 15
            periodSeconds: 20
            failureThreshold: 3

          # Resource limits: prevent a runaway container from affecting other Pods
          resources:
            requests:
              cpu: "100m"      # 0.1 CPU cores (minimum guaranteed)
              memory: "256Mi"
            limits:
              cpu: "500m"      # 0.5 CPU cores (maximum allowed)
              memory: "512Mi"

      # Security context: enforce non-root at pod level
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000

k8s/service.yaml

apiVersion: v1
kind: Service
metadata:
  name: payments-api
spec:
  selector:
    app: payments-api        # routes to Pods with this label
  ports:
    - protocol: TCP
      port: 80               # port exposed by the Service
      targetPort: 8000       # port the container listens on
  type: ClusterIP            # internal-only (use LoadBalancer or Ingress for external)
---
# HPA: automatically scale based on CPU utilization
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: payments-api-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: payments-api
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70   # scale up when average CPU > 70%

Readiness vs. Liveness Probes: Get These Right

These are commonly confused and wrongly configured. Readiness probe: "Is this Pod ready to receive traffic?" A Pod that fails readiness is removed from the Service's endpoint list — no traffic is sent to it, but it is not restarted. Use this for startup time and temporary unreadiness (e.g., a cache is being warmed). Liveness probe: "Is this Pod alive and worth keeping?" A Pod that fails liveness is killed and restarted. Use this for detecting deadlocks or stuck processes. The disaster scenario: setting the liveness probe to fail under high load causes Kubernetes to kill Pods under load, making the overload worse. Set liveness thresholds conservatively; readiness thresholds responsively.

kubectl command	What it does	When to use
`kubectl get pods`	List Pods and their status	First thing when something seems broken
`kubectl describe pod <name>`	Events, conditions, resource usage for a pod	Diagnosing crash loops, scheduling failures
`kubectl logs <pod> -f`	Stream logs from a container	Investigating application errors
`kubectl exec -it <pod> -- sh`	Open a shell inside a running container	Debugging network, filesystem, runtime issues
`kubectl apply -f manifest.yaml`	Apply declarative config; create or update objects	Deploying changes
`kubectl rollout status deployment/<name>`	Watch rolling update progress	Monitoring a deployment rollout
`kubectl rollout undo deployment/<name>`	Roll back to the previous revision	Emergency rollback after a bad deploy
`kubectl top pods`	Real-time CPU/memory usage per Pod	Resource investigation; sizing

Consulting Lens: The Kubernetes Adoption Conversation

Kubernetes is powerful but complex. Not every client needs it. The right framing: Kubernetes makes sense when (1) you run many services and need consistent deployment, scaling, and self-healing; (2) you need fine-grained resource management across a shared compute pool; (3) you want platform-level abstractions (namespaces, RBAC, network policies) that a team of teams can use consistently. It does not make sense when you are running one or two services, or when operational complexity would outpace the team's Kubernetes expertise. Managed Kubernetes (EKS, GKE, AKS) dramatically reduces the operational overhead — but even managed K8s requires understanding the concepts covered here to debug production issues. The honest position: Kubernetes is not magic — it trades application-level operational problems for platform-level operational problems that require a different skill set.

WHY the declarative bet The declarative model (desired state, not imperative commands) is why Kubernetes is powerful and why it is complex. You describe the goal; Kubernetes figures out the steps. This works at scale but requires understanding the control loop to reason about failure modes.

HOW the four primitives Pod (run a container), Deployment (maintain N pods, rolling updates), Service (stable network endpoint), ConfigMap/Secret (inject configuration). These four primitives handle 80% of a real workload.

WHERE production at scale Kubernetes is the dominant production runtime for containers. EKS, GKE, and AKS eliminate node management; the application team still owns manifests, probes, resource limits, HPA configuration, and RBAC — the concepts covered in this lesson.

Knowledge check

InterviewWhat is the difference between a liveness probe and a readiness probe in Kubernetes, and what happens when each fails?

They are synonyms — both cause the Pod to be restarted when they fail.
Readiness failure removes the Pod from Service routing (no traffic sent) but does not restart it — it is temporarily unavailable. Liveness failure triggers a Pod restart — the container is killed and replaced. Readiness handles temporary unreadiness; liveness handles genuinely broken processes.
Liveness failure removes the Pod from routing; readiness failure restarts it.
Both cause the node to be replaced, not the Pod.

Answer

Readiness failure removes the Pod from Service routing (no traffic sent) but does not restart it — it is temporarily unavailable. Liveness failure triggers a Pod restart — the container is killed and replaced. Readiness handles temporary unreadiness; liveness handles genuinely broken processes.

Readiness: "should we send traffic to this Pod right now?" Failing readiness is like a "back in 5 minutes" sign on a store — the pod is still alive, just temporarily not accepting work. It is removed from the Service's endpoint list but not restarted. Useful for: startup warm-up, graceful degradation, temporary resource exhaustion. Liveness: "is this Pod still functioning at all?" Failing liveness is like a smoke detector alarm — the container is assumed broken and is restarted. Useful for: detecting deadlocks, hung threads, memory corruption. The critical mistake is making liveness too sensitive — it should only fail on genuinely unrecoverable states, not on high load.

Knowledge check

InterviewA client's Kubernetes Deployment is configured with maxUnavailable: 0 and maxSurge: 1 for rolling updates. What does this mean in practice for zero-downtime deployments?

It means only one Pod can run at a time, causing brief downtime during updates.
maxUnavailable: 0 means no Pod is terminated until its replacement is ready and passing the readiness probe. maxSurge: 1 allows one extra Pod to be created temporarily. Together: roll out by creating the new Pod, wait until it passes readiness, then terminate one old Pod — fully zero-downtime.
These settings are only relevant to blue-green deployments, not rolling updates.
maxSurge: 1 means the update will fail if more than 1 Pod crashes during rollout.

Answer

maxUnavailable: 0 means no Pod is terminated until its replacement is ready and passing the readiness probe. maxSurge: 1 allows one extra Pod to be created temporarily. Together: roll out by creating the new Pod, wait until it passes readiness, then terminate one old Pod — fully zero-downtime.

The rolling update strategy controls how Pods are replaced. maxUnavailable: 0 means the replica count never drops below the desired number during the update — not a single Pod is removed before a healthy replacement exists. maxSurge: 1 allows one additional Pod above the desired count, so the replacement can be started before the old one is removed. The sequence: desired = 3 replicas; start 1 new Pod (now 4 total); wait for readiness probe to pass; terminate 1 old Pod (back to 3); repeat. With maxSurge: 0 and maxUnavailable: 1, you'd terminate first then start — a brief capacity reduction. The maxUnavailable: 0, maxSurge: 1 pattern is the safe default for production services.

Exercise

A FastAPI service running in Kubernetes starts crashing frequently. Your SRE colleague sends you this kubectl output. Identify all possible causes for each indicator, and describe what kubectl commands you would run next to narrow down the problem:

NAME                           READY   STATUS             RESTARTS   AGE
payments-api-7d9b6c8f4-xk2p9   0/1     CrashLoopBackOff   8          12m
payments-api-7d9b6c8f4-mn3q1   1/1     Running            0          45m
payments-api-7d9b6c8f4-pz8r7   0/1     OOMKilled          1          5m

Show solution

OBSERVATIONS:
- Pod xk2p9: CrashLoopBackOff, 8 restarts → application crashes at startup
  (bad env var / secret? missing dependency? failed DB connection?)
- Pod mn3q1: Running fine → the issue is intermittent or config-related
- Pod pz8r7: OOMKilled → container exceeded memory limit and was killed by kernel

LIKELY CAUSES:
1. OOMKilled (pz8r7): memory limit set too low in resources.limits.memory
   OR a memory leak in the application
2. CrashLoopBackOff (xk2p9): application throws an uncaught exception on startup;
   could be: DATABASE_URL secret missing or wrong; import error; bad config value

INVESTIGATION COMMANDS:
# 1. Check logs for the crashing pod (most recent crash)
kubectl logs payments-api-7d9b6c8f4-xk2p9 --previous
# The --previous flag shows logs from the last terminated container

# 2. Describe the crashing pod for events and last state
kubectl describe pod payments-api-7d9b6c8f4-xk2p9
# Look for: Last State (exit code), Events (OOMKilled message, image pull errors)

# 3. Check resource usage on the running pod
kubectl top pod payments-api-7d9b6c8f4-mn3q1
# If near the memory limit, it may OOMKill next

# 4. Verify secret exists and has the right key
kubectl get secret payments-secrets -o jsonpath='{.data.database-url}'
# (value is base64 encoded; pipe to base64 -d to decode)

# 5. If startup error, exec into a working pod to test the DB connection
kubectl exec -it payments-api-7d9b6c8f4-mn3q1 -- python -c \
  "import os; print(os.environ.get('DATABASE_URL', 'NOT SET'))"

NEXT ACTIONS:
- If OOMKilled: increase memory limit or investigate memory leak with profiler
- If startup crash: fix the root cause (missing secret, bad config)
- After fix: kubectl rollout restart deployment/payments-api to replace all pods

Key Takeaways

Kubernetes uses a declarative, desired-state model: declare what you want, the control loop makes it happen.
Pod = unit of execution; Deployment = manages Pod replicas and updates; Service = stable network endpoint.
Readiness probe: remove from routing (no restart). Liveness probe: kill and restart. Configure both carefully.
Resource requests/limits enable bin-packing and prevent noisy-neighbor problems; OOMKilled means a limit was hit.
Rolling updates with maxUnavailable: 0, maxSurge: 1 achieve zero-downtime deployments.
kubectl logs, describe, exec, and rollout are the four essential debugging commands.