During testing you point your frontend straight at a backend Pod at 10.244.1.7, and it works fine. A week later the frontend stops loading data. What most likely happened?

That Pod was replaced, and the replacement came up on a different IP, so 10.244.1.7 now reaches nothing. Pods are disposable and every replacement gets a fresh address, exactly like the jump to 10.244.2.15 you watched earlier. That is the whole reason you dial a Service's stable name instead of a raw Pod IP.

In kubectl expose deployment web --port=80 --target-port=8080, what do the two port numbers mean?

port is what callers dial on the Service; targetPort is the port the container listens on inside the Pod, and traffic arriving on 80 is forwarded to 8080. The Service answers on port and forwards to targetPort, and the two numbers are free to differ.

You create a Service, but kubectl get endpoints shows . The Pods you meant it to pick up are Running and passing their readiness probe. What is the likeliest cause?

The Service's selector label doesn't match the Pods' label exactly, say app: webs against app: web. With healthy Pods, an empty endpoint list nearly always means the selector matches nothing.

Why pods need a Service

Pods move; a Service stays put.

Beginner10 min · lesson 16 of 24

In plain terms

Pod IPs change. A Service IP and DNS name do not.

The frontend needs the backend. For reliability you run three identical backend copies, so if one dies the others still serve. Which of the three does the frontend call? Each copy is a Pod. Pods are disposable: an update, a crash, or a scale-down deletes one and starts another, and the replacement gets a new IP (Internet Protocol address). Hard-coding a pod IP in the frontend will break the next replacement.

A Pod IP works until that Pod is replaced, then it belongs to nothing. Do not wire one part of the app to another by Pod IP. Hard-code a backend Pod address into the frontend, let a rollout swap that Pod, and the frontend calls a dead address. It will break. It tends to break at 2 a.m.

You don't have to take any of that on faith. Start the backend, then list the Pods with -o wide, a flag that adds each Pod's IP and the node it landed on to the usual output.

terminal

$ kubectl get pods -o wide
NAME                  READY   STATUS    RESTARTS   AGE   IP           NODE
web-6b8f4c9d7-4xk2p   1/1     Running   0          2m    10.244.1.7   node-a
web-6b8f4c9d7-8lprd   1/1     Running   0          2m    10.244.2.4   node-b
web-6b8f4c9d7-r9m2t   1/1     Running   0          2m    10.244.3.9   node-c

Three copies, three IPs. Say the frontend stored the first one, 10.244.1.7. Delete that Pod the way a rollout or a crash would, and watch the Deployment stand up a replacement.

terminal

$ kubectl delete pod web-6b8f4c9d7-4xk2p
pod "web-6b8f4c9d7-4xk2p" deleted
 
$ kubectl get pods -o wide
NAME                  READY   STATUS    RESTARTS   AGE   IP            NODE
web-6b8f4c9d7-q7ntz   1/1     Running   0          9s    10.244.2.15   node-b
web-6b8f4c9d7-8lprd   1/1     Running   0          4m    10.244.2.4    node-b
web-6b8f4c9d7-r9m2t   1/1     Running   0          4m    10.244.3.9    node-c

Same app, new Pod, new IP: 10.244.2.15. The address your frontend was holding, 10.244.1.7, now belongs to nothing at all, so a call to it hangs until it gives up.

terminal

$ curl 10.244.1.7:8080
curl: (28) Failed to connect to 10.244.1.7 port 8080 after 133024 ms: Connection timed out

A Service is the desk number that never changes

A Service sits in front of a group of Pods and gives them one stable name and one stable IP for as long as the Service exists. It selects Pods by label, for example app: web. Matching Pods join the set. Incoming requests go to one of the healthy Pods so load is spread. That spreading is load balancing.

A Service is the stable front door

The Service keeps its own live list of the healthy Pods behind it. Pods come and go. The name your frontend dials never moves.

The fastest way to put a Service in front of Pods you already have is a single command run through kubectl, the command-line tool you use to send instructions to a Kubernetes cluster (the group of machines Kubernetes runs your apps on). Those Pods are usually looked after by a Deployment. A Deployment is a controller, meaning a small background program that watches your app and keeps the right number of Pod copies alive. The command below reads that Deployment's labels and builds a matching Service for you.

terminal

$ kubectl expose deployment web --port=80 --target-port=8080
service/web exposed

Two port numbers turned up there, and they trip almost everybody at first. port is the number the Service answers on, the one other apps dial. targetPort is the port your container actually listens on inside the Pod. The two are allowed to differ, and often should, which is half of why people muddle them. Traffic arrives on port 80 and gets forwarded to 8080. Ask Kubernetes for the Service now, and you'll see the permanent address it handed you.

terminal

$ kubectl get service web
NAME   TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
web    ClusterIP   10.96.211.34   <none>        80/TCP    9s

That CLUSTER-IP value, 10.96.211.34, is the steady address the Service claimed, and it stays put for the whole life of the Service. Notice it sits in a different range from the Pod IPs you watched change. A Service address is handed out once and never recycled while the Service lives. The TYPE column reads ClusterIP, which means the address works from inside the cluster. Running a command is fine for a quick test, but real projects keep their setup in files they can review and re-apply. Here's the same Service written as a YAML file (YAML is a plain-text format for describing configuration, using indentation instead of brackets). This one is complete. Save it and run kubectl apply -f svc.yaml exactly as it stands.

svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  selector:
    app: web            # send traffic to Pods wearing this label
  ports:
    - port: 80          # the Service answers here
      targetPort: 8080  # forwarded to this port inside the Pod

Apply the file, then run the one command that tells you whether the Service actually found any Pods. get endpoints lists the live Pod addresses the Service is sending traffic to right now. If the labels line up, you get one address per running Pod, and the replacement Pod from earlier turns up on its own with no editing from you.

terminal

$ kubectl apply -f svc.yaml
service/web configured
 
$ kubectl get endpoints web
NAME   ENDPOINTS                                           AGE
web    10.244.2.15:8080,10.244.2.4:8080,10.244.3.9:8080   1m

What the Service quietly handles

Once that Service exists, three headaches go away. Identity is the first: other apps get a fixed name and IP they can trust for good, even as the Pods behind it are swapped out. Balance is the second, which you already saw, spreading requests so that scaling up adds real capacity instead of idle copies. The third is the one people underrate. The Service keeps its own membership list. When a new Pod passes its health check and goes live, it joins by itself. When a Pod dies or scales away, it drops off. You edit nothing. The list you see in get endpoints reshapes itself as your Pods come and go.

The address that belongs in config is the DNS name. DNS (Domain Name System) turns names into IPs. From inside the cluster you can call web, or the full web.default.svc.cluster.local. Behind that name, kube-proxy (or whichever data plane the cluster uses) programs every node so traffic to the Service lands on a current endpoint. Names in config, not Pod IPs.

An empty endpoint list has two very different causes, and from the caller's seat they look identical. Either the selector matches no Pod at all, or the Pods exist but none of them is Ready. Your client gets the same symptom either way, a request that goes nowhere and eventually times out, so the timeout tells you nothing about which situation you're in. The endpoints list is where the two stories come apart.

This is why the rule exists at all. A frontend with a backend Pod IP baked into its config will sail through every test you throw at it, then die on the first rollout, at whatever hour that rollout happens to run. Point it at the Service instead and the rollout stops being an event.

Try this

Run it yourself. Put a ClusterIP Service in front of a couple of Pods, curl it from a separate Pod using the cluster DNS name, then delete a backend Pod and check that the Service keeps answering through the replacement.

terminal

$ kubectl create deployment api --image=nginx:1.27 --replicas=2
deployment.apps/api created
$ kubectl expose deployment api --port=80 --target-port=80
service/api exposed
 
$ kubectl get svc api; kubectl get endpoints api
NAME   TYPE        CLUSTER-IP     PORT(S)   AGE
api    ClusterIP   10.96.45.12    80/TCP    5s
NAME   ENDPOINTS                         AGE
api    10.244.1.40:80,10.244.1.41:80     5s
 
$ kubectl run curl --image=busybox:1.36 --restart=Never --command -- sleep 300
pod/curl created
$ kubectl exec curl -- wget -qO- http://api.default.svc.cluster.local | Select-String -Pattern 'Welcome|nginx' | Select-Object -First 1
Welcome to nginx!
 
$ kubectl delete pod curl; kubectl delete svc api; kubectl delete deployment api
pod "curl" deleted
service "api" deleted
deployment.apps "api" deleted

Takeaway

Carry one habit out of this lesson. When something inside the cluster can't reach a Service, run kubectl get endpoints before you blame the network. Addresses on that list mean traffic is being handed somewhere real and the fault lies further along. An empty list means the selector or readiness is at fault, and no amount of staring at packets will ever show you that.

Empty endpoints? Look at the labels first

If kubectl get endpoints comes back with nothing, the Service matched no ready Pod. Two causes account for nearly all of it. Either the label in the Service's selector doesn't match the Pods' label exactly (a typo like app: web against app: webs), or the Pods are running but failing their readiness probe, the health check that decides whether a Pod is fit to receive traffic. Run kubectl describe service web and read the Endpoints line. If it says <none>, look at the selector. A Pod that isn't ready is left off the list on purpose, so it never gets handed a request it can't answer.

Secrets: handling passwordsThe kinds of Service

Cluster architectureKubernetes administration Security domains & threat modelKubernetes security & hardening The cluster attack surfaceKubernetes attack & defense kubectl cheat sheetCheat sheet Kubernetes interview questionsInterview prep

Why pods need a Service

A Service is the desk number that never changes

What the Service quietly handles

Try this

Takeaway

Related