>_SecOpsLog_
CoursesKubernetes administrationServices

Services

Stable identity and load balancing for pods.

Intermediate12 min · lesson 33 of 65
In plain terms
A Service is a company’s main phone number. You always dial the same number, and it quietly routes you to whichever employee is free — you never need to know anyone’s direct desk extension.

Pods don't keep their addresses. A Pod (the smallest thing you can deploy in Kubernetes, one or more containers that share one network identity) gets an IP address when it starts and loses it the moment it dies. An IP address is just a number on the network, the way a house has a street number. Roll out a new version of a Deployment (the object that manages a batch of identical Pods and swaps them out on every update) and every old Pod IP is thrown away and replaced with fresh ones. So if one app hard-codes the IP of another, that link snaps the first time the other side restarts, scales, or gets moved to a different node, meaning a different worker machine. A Service is how Kubernetes makes a moving target hold still.

Think of a big office with a front desk. You call and ask for Sales. You never learn anyone's desk extension, and you don't care who picks up, you just want someone in Sales who's free right now. People in that department move desks, take leave, and get hired, and none of it changes the number you dial. A Service is that front desk. Clients ask for a name, the Service hands them off to whichever healthy Pod is available behind it, and the Pods can churn all day without a single client noticing.

A stable front door for a moving target

A Service is a small object built around a label selector. A label is a key/value tag you stick on Pods, like app: web. A selector is a standing search for that tag. Give the Service the selector app: web and Kubernetes hands you two things that never change: a ClusterIP (a stable virtual IP, handed out once by the API server, the control plane's front door, from the cluster's pool of service addresses) and a name that other Pods can look up. That lookup runs through DNS, the Domain Name System, the internet's phonebook that turns a readable name into an address. Inside the cluster, CoreDNS (the built-in name service) answers web.default.svc.cluster.local, or just web from inside the same namespace, which is Kubernetes' folder-like way of grouping objects. The selector is the whole trick. The Service doesn't name specific Pods, it describes them, and Kubernetes keeps the membership list current for you as Pods come and go.

web.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 3
  selector:
    matchLabels: { app: web }
  template:
    metadata:
      labels: { app: web }
    spec:
      containers:
        - name: web
          image: hashicorp/http-echo:1.0
          args: ["-listen=:8080", "-text=hi"]
          ports:
            - { name: http, containerPort: 8080 }
          readinessProbe:
            httpGet: { path: /, port: http }
---
apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  selector: { app: web }
  ports:
    - port: 80          # clients hit web:80
      targetPort: http  # forwarded to the container's named port (8080)
create it and look at what you got
kubectl apply -f web.yaml
kubectl get svc web
kubectl get endpointslices -l kubernetes.io/service-name=web
output
deployment.apps/web created
service/web created
 
NAME   TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
web    ClusterIP   10.96.142.30    <none>        80/TCP    9s
 
NAME        ADDRESSTYPE   PORTS   ENDPOINTS                          AGE
web-x7k2p   IPv4          8080    10.244.1.7,10.244.2.4,10.244.1.9   9s

Three Pods, three IP addresses on the roster, one Service fronting them at 10.96.142.30 and at the name web. Nothing you deploy later has to know those Pod IPs. It talks to web and lets the Service do the routing.

The live roster behind the Service

Here's what's actually happening behind the desk. Picture a roster, the sign-in sheet that says who's on shift and reachable right now. A controller in the control plane, the EndpointSlice controller (part of kube-controller-manager, the process that runs Kubernetes' built-in background controllers), watches Pods and Services. Every time a Pod that matches the selector becomes Ready, the controller writes that Pod's IP into an EndpointSlice object. Delete the Pod and its entry goes away. Let it stay up but fail its readiness probe, and the entry stays too: the controller flips that endpoint's ready condition to false, and kube-proxy is what then refuses to send it traffic. So the roster is a sign-in sheet with a ready flag beside each name, not just a list of names. The Service itself stores almost nothing about who's behind it. The EndpointSlice is what kube-proxy reads to program the actual routing. kube-proxy is the small network agent running on every node. EndpointSlice is the modern form of this roster. The older single Endpoints object still exists for compatibility, but slices scale better, because a busy Service spreads its members across several small objects instead of one giant one.

see the roster the Service routes to
kubectl describe svc web
output
Name:              web
Namespace:         default
Selector:          app=web
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.96.142.30
Port:              <unset>  80/TCP
TargetPort:        http/TCP
Endpoints:         10.244.1.7:8080,10.244.2.4:8080,10.244.1.9:8080
Session Affinity:  None
Events:            <none>

Readiness is the gate that keeps this safe. A Pod can be Running and still be kept off the roster, because Running only means the container process started, while Ready means its readiness probe passed. A readiness probe is a health check Kubernetes runs against the Pod, like knocking on the door to see if anyone answers. Only Ready Pods get traffic. That's how a rolling update never sends a request into a Pod that's still warming up, and how a Pod that starts failing its probe gets pulled out of rotation within seconds without anyone deleting it. Pulled out of rotation, though, not struck off the sheet. The Endpoints line above lists ready addresses only, so a failing Pod drops off that line while kubectl get endpointslice web-x7k2p -o yaml still shows its address, marked ready: false.

port, targetPort, and the four ways in

Two port numbers trip people up, because they look interchangeable and aren't. port is the number clients dial on the Service (web:80). targetPort is the port on the container that traffic is forwarded to (8080 here). They're allowed to differ, and usually should, so you can keep a clean public port while the app listens wherever it likes. Point targetPort at a named container port (http above) instead of a raw number and the Service stops keeping its own copy of the number: change containerPort in the Pod spec and the Service follows, with no second file to remember. That closes the gap between the Service and the Pod spec. It does nothing about the gap between the Pod spec and the process actually running, because a container's ports: block is a label, not a promise, and nothing verifies it.

One Service object, four ways to be reached, and they are not four separate boxes you pick one from. ClusterIP is the default and gives an internal-only virtual IP, right for almost all Pod-to-Pod traffic inside the cluster. NodePort opens the same high-numbered port (30000 to 32767 by default) on every node and forwards it inward, a blunt way to reach a Service from outside when you have no cloud, and it keeps a ClusterIP as well. LoadBalancer asks the cloud provider to set up a real external load balancer pointing at the Service, the standard front door on a managed cluster, and underneath it gets both a NodePort and a ClusterIP, which is how the cloud balancer reaches your nodes at all. That is why kubectl get svc still prints a CLUSTER-IP next to a LoadBalancer, and it is not a bug. Headless is the odd one out and is not a fourth type: spec.type stays ClusterIP and you set clusterIP: None, which skips the virtual IP and makes DNS return the individual Pod IPs, which is what StatefulSet members want. A StatefulSet is a workload whose Pods keep stable identities, the way database replicas do. For many HTTP apps you don't hand each one its own load balancer at all. You put an Ingress in front of plain ClusterIP Services. An Ingress is a single HTTP doorway that routes by hostname or URL path, and it's covered later in this section.

Which Service type do you need?
How is this reached?
one Service object, four exposure modes
only from inside the cluster
ClusterIP (default)
stable internal virtual IP + DNS name; the right pick for almost everything
clients must address individual Pods
Headless (clusterIP: None)
DNS returns Pod IPs directly; used by StatefulSets
from outside, on a cloud
LoadBalancer
provider provisions a real external load balancer pointing at the Service
from outside, no cloud LB
NodePort
same high port on every node; crude but works on bare metal
Start at ClusterIP and only move outward when something genuinely needs external reach. NodePort and LoadBalancer sit on top of ClusterIP rather than replacing it, and headless is a ClusterIP Service with clusterIP: None. For many HTTP services, one Ingress in front of several ClusterIP Services beats a LoadBalancer per app.

When the front desk answers but the line is dead

A Service is reported down. Before you touch anything, split it into two questions asked in order. Is the roster empty? And if it isn't, does the roster point where the app actually listens? Start with the roster, because it's one command and it decides everything. kubectl describe svc (or kubectl get endpointslices) tells you instantly whether any Pod sits behind the Service. Empty endpoints mean the Service is fine and the real problem is upstream: either no Pod carries the label the selector wants, or Pods carry it but are stuck not-Ready. Both show up in seconds.

diagnose an unreachable Service
# clients hitting web get 'connection refused'. who's on the roster?
kubectl describe svc web | grep -i endpoints
 
# empty roster: do the Pods actually carry the label the selector wants?
kubectl get pods -l app=web
kubectl get pods --show-labels | grep web
output
Endpoints:         <none>
 
No resources found in default namespace.
 
web-6d4c8f9b7-abcde   1/1   Running   0   5m   app=web-app,pod-template-hash=6d4c8f9b7

There it is. The Pods are healthy (1/1 Running), but they're labeled app=web-app while the Service selects app=web, so the selector matches nothing and the roster stays empty. Fix whichever side is wrong and the endpoints populate within a second. If instead the Pods matched but showed 0/1, you'd go read the readiness probe or the container logs for why they never go Ready. And once the roster fills but traffic still won't flow, you've crossed out of Service territory into how the kernel actually rewrites ClusterIP packets onto a real Pod, which is kube-proxy's job and the next lesson.

Full roster, still refused: the targetPort trap
A Service can show a healthy roster and still refuse every connection if targetPort points at a port the container isn't listening on. The Service forwards web:80 to, say, 8080, but the app is actually bound to 3000, so the kernel delivers the packet and the Pod resets it. The symptom looks exactly like a broken Service, but kubectl describe svc shows populated Endpoints, which already rules out selector and readiness. Confirm it by curling a Pod's real IP and port directly from another Pod: if podIP:8080 is refused too, the mismatch is your targetPort, not the Service. A named container port with targetPort: http (as in the manifest above) keeps the Service and the Pod spec in step, but it cannot catch this one. Nothing checks that the container really listens on the port its own spec names, so if the app moves to 3000 while containerPort still says 8080, the name http still resolves to 8080 and the connection is still refused. Check the manifest against what the process binds, not against itself.

ClusterIP, NodePort, and LoadBalancer are exposure modes, not different apps. Start with ClusterIP inside the cluster.

sessionAffinity is sticky and can hide bad replicas. Prefer fixing readiness.

headless Services return pod IPs for StatefulSets and similar. That is DNS design, not a missing ClusterIP bug.

Try this

Apply web.yaml, then curl the Service from a throwaway client Pod, because the http-echo image carries no shell and no curl of its own. Once you get a reply, point the Service's selector at a label no Pod carries. The roster empties, the same curl comes back refused, and putting the selector back fills the roster again within a second.

terminal
$ kubectl apply -f web.yaml
 
# http-echo has no shell, so bring your own client Pod
$ kubectl run client --rm -it --image=curlimages/curl:8.5.0 --restart=Never -- curl -s http://web
hi
pod "client" deleted
 
# break it on purpose: aim the selector at a label no Pod carries
$ kubectl patch svc web -p '{"spec":{"selector":{"app":"web-app"}}}'
service/web patched
 
$ kubectl describe svc web | grep -i endpoints
Endpoints:         <none>
 
$ kubectl run client --rm -it --image=curlimages/curl:8.5.0 --restart=Never -- curl -s http://web
curl: (7) Failed to connect to web port 80 after 1 ms: Connection refused
pod "client" deleted
 
# put the selector back and the roster refills
$ kubectl patch svc web -p '{"spec":{"selector":{"app":"web"}}}'
service/web patched

Takeaway

Services give stable virtual IPs and DNS in front of changing pods. Endpoints follow ready pods that match the selector.

Quick check
01Your Deployment's Pods all show 1/1 Running, but kubectl describe svc web prints Endpoints: <none> and clients get 'connection refused'. What's the most likely cause, and the fastest confirming check?
Correct — Empty endpoints on healthy-looking Pods is almost always a label/selector mismatch or Pods that are Running but not Ready. The roster is built only from Ready Pods whose labels match, and one command confirms both.
Incorrect — With an empty roster there is nothing for kube-proxy to route to, so its state is irrelevant. Empty endpoints is a selector/readiness problem upstream of kube-proxy, and restarting it wastes time.
Incorrect — A 'connection refused' to a ClusterIP means the name already resolved to an IP. A DNS failure looks like 'bad address' or 'could not resolve host', not a refused connection.
Incorrect — The Service already has a ClusterIP (describe shows one), so allocation succeeded. Range exhaustion blocks creating new Services; it does not empty an existing Service's endpoints.
02You need each Pod of a StatefulSet to be individually addressable by name, so a replica can reach the primary on purpose rather than a random backend. Which Service type gives you that?
Incorrect — NodePort exposes one Service to the outside on a high port; it still load-balances across Pods rather than naming them individually.
Incorrect — LoadBalancer adds an external front door but still hides the individual Pods behind one address.
Incorrect — A normal ClusterIP hands out one shared virtual IP and balances across Pods; it does not expose them by name.
Correct — a headless Service skips the virtual IP and makes DNS return each Pod's address, which is how StatefulSet members get stable per-Pod names.
03kubectl describe svc web shows three populated Endpoints, yet clients still get 'connection refused.' Curling a backing Pod's own IP on the Service's targetPort is refused too. What's the fault?
Incorrect — A selector mismatch leaves Endpoints empty, but here the roster is populated, so the selector is already matching.
Correct — populated Endpoints rule out selector and readiness, and a Pod IP refusing the port directly means the app is bound to a different port than targetPort names.
Incorrect — A refused connection straight to a Pod's own IP bypasses kube-proxy entirely, so its state can't be the cause.
Incorrect — A DNS failure reads as 'could not resolve host,' not a refused connection to an IP you have already reached.
Pod networking & CNIService networking & kube-proxy

Related

What Kubernetes isKubernetes fundamentalsSecurity domains & threat modelKubernetes security & hardeningThe cluster attack surfaceKubernetes attack & defenseKubernetes interview questionsInterview prep