You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

History

Samuel Vaillant b203b3be92 fix(probes): http -> tcp (#56 )		4 years ago
..
01-liveness-probe.yml	Fix exit not found error in liveness probe manifest (#38 )	5 years ago
02-readiness-probe.yml	Fix exit not found error in readiness probe manifest (#40 )	5 years ago
README.md	fix(probes): http -> tcp (#56 )	4 years ago

README.md

Unescape Escape

Liveness and readiness probes, and how it impacts your pods

Introduction

To be able to handle pods correctly, Kubernetes needs to know how they are behaving. For this it needs to have a way to know the state of the containers running inside a given pod. The creator of Kubernetes decided to go to a "black box" testing. "Black box" testing means that Kubernetes doesn't interact directly with the containers. It is each container that says how Kubernetes can know its state.

You can define two probes for Kubernetes to know the state of your container: "liveness" and "readiness" probes.

Liveness probe

The liveness probe is here to detect if a container is still alive. Meaning, if the container is not in a broken state, in a dead lock, or anything related. This is always useful. It helps Kubernetes to know if your container is alive or not and so it can take decision based on that, like restarting it.

Readiness probe

The readiness probe is here to detect if a container is ready to serve traffic. It is useful to configure when your container will receive external traffic sent by Kubernetes. Most of the time, when it's an API.

Defining a probe

Both liveness and readiness probes have the same configuration. You have three ways of configuring a probe:

exec probe
http probe
tcp probe

Exec probe

The exec probe lets you configure a command that Kubernetes will run in your container. If the command exits with a non zero status the probe will be considered unhealthy:

livenessProbe:
  exec:
    command:
    - cat
    - /tmp/healthy
  initialDelaySeconds: 5
  periodSeconds: 5

The exec probe contains only a command, which is mandatory and represents which command it should run. Here it will be cat /tmp/healthy. We will see later what initialDelaySeconds and periodSeconds means.

HTTP probe

The http probe lets you configure a HTTP endpoint that Kubernetes will call in your container. If this endpoint returns a non 2XX status the probe will be considered unhealthy:

livenessProbe:
  httpGet:
    path: /healthz
    port: 8080
    httpHeaders:
    - name: Custom-Header
      value: Awesome
  initialDelaySeconds: 3
  periodSeconds: 3

The http probe has two mandatory fields path and port and one optional httpHeaders.

path: lets you configure which http path the probe should call.
port: lets you configure which port the probe should connect to.
httpHeaders: lets you configure http headers the probe should send with its call.

TCP probe

The tcp probe lets you configure a TCP port that Kubernetes will try to connect to. If it does not manage to establish a connection the probe will be considered unhealthy:

livenessProbe:
  tcpSocket:
    port: 8080
  initialDelaySeconds: 15
  periodSeconds: 20

The tcp probe has one mandatory fields port. It represents which TCP port Kubernetes will try to connect to.

`initialDelaySeconds` and `periodSeconds`

The periodSeconds field specifies that Kubernetes should perform the probe every N seconds. The initialDelaySeconds field tells Kubernetes that it should wait N second before performing the first probe.

If we take the example:

livenessProbe:
  httpGet:
    path: /healthz
    port: 8080
  initialDelaySeconds: 3
  periodSeconds: 5

This probe will wait 3 seconds before doing the first probing. The probing will be an http call to http://localhost:8080/healthz. After the first wait of 3 seconds, each probing will be done every 5 seconds.

Impact of probes on Kubernetes

Liveness probe impact

Look and apply the file 01-liveness-probe.yml.

Run kubectl get pods -w and see what is happening.

The liveness of this pod fails (exit 1), so Kubernetes detects that the pod is not alive anymore and restarts it. So this probe is key for Kubernetes to know if a pod should be restarted or not.

Readiness probe impact

Look and apply the file 02-readiness-probe.yml.

Run kubectl get pods -w and see what is happening. Run kubectl get deployments -w and see what is happening.

The pods are "Running" but not ready (the READY column to 0/1).

kubectl get pods
NAME                                    READY   STATUS    RESTARTS   AGE
readiness-deployment-5dd7f6ff87-jsm9f   0/1     Running   0          2m17s
readiness-deployment-5dd7f6ff87-wnrmg   0/1     Running   0          2m17s

If you try to access the service readiness-service, with kubectl port-forward service/readiness-service 8000:80. It won't work. Kubernetes sees that all the pods are not ready, so it won't send traffic to them.

The readiness probe is also used when you do rolling updates. Kubernetes will wait for the pods with the new version to be ready before sending traffic to it.

Good practices

Most of time having the readiness and liveness probe to be the same is enough. In some cases you might want to them to be different. A good example is a container running a mono-threaded application that accepts HTTP calls (who said PHP). Let’s say you have an incoming request that will very long to be processed. Your application is not able to receive any other request, as it’s blocked by the incoming requests. So it’s not “ready”; on the other hand it’s processing a request so it’s “alive”.

Another tip, your probes should not call dependent services of your application, to prevent cascading failure.

Clean up

kubectl delete service,deployment,pod --all

README.md Unescape Escape