Answers: <p>1 - yes, containers are left there forever. Think of readiness probe as a breaker, if the app gets overwhelmed it cuts the traffic to the pod until it's ready to get traffic again.</p>
  <p>2 - no, by itself, readiness probes only stop/start traffic to a pod. To signal kubernetes that the pod needs to be restarted use liveness probes.</p>
  <p>3 - yes, as per answer 1. Readiness probes are a way of letting kubernetes know that the pod can handle traffic or if they need a break.</p>
  <p><strong>probe: startup probes - waits for the app to start</strong></p>
  <p>use case: apps that need some time before start serving traffic, for example, building assets on start, pre-fetching assets, building caches, etc.</p>
  <p><strong>liveness probes - checks if the app is healthy</strong></p>
  <p>use case: check if the app is running, if the app has exited due to an exception and needs to be restarted, or if the app is running but can't serve requests</p>
  <p><strong>readiness probes - checks if the app can handle traffic</strong></p>
  <p>use case: apps that can arbitrarily stop receiving traffic, e.g. maintenance mode, data intensive apps that can degrade in performance and need a way to</p>
  <p>You should use liveness and readiness probes always, use startup probes when your app takes more time than usual to start.</p>
  <p>Example:</p>
  <ul>
  <li>python app with postgres database:</li>
  <li>liveness probe: get /health status 200</li>
  <li>readiness probe: get /health status 200, timeout 1s</li>
  </ul>
  <p>the /health endpoint queries the database using a simple query (select id, email from user limit 1) and returns a json</p>
  <p>if the pod loses connectivity to the database, the liveness probe will fail and the pod will be restarted.
  if the database is being overwhelmed by requests, the query will take more than 1s, the probe will mark the pod as not ready, preventing that additional requests reach the database and preventing it from progressing from overwhelmed to crashed. once the database is able to catch up and starts returning queries in less than 1s, readiness probe will mark the pod as healthy again and kubernetes will resume serving requests to the pod.</p>
  
Answers: CC BY-SA 4.0
Answers: 2021-11-15T12:32:32.553Z
Answers: Thanks for the explanation! However I read that *liveness* probe shouldn't test a different component (like the database), because a failure in one component can damage the availability of entire application (with continuous restarts)
Answers: collimarco
Answers: CC BY-SA 4.0
Answers: 2021-11-15T19:26:05.267Z
Answers: yup! modern applications rely on several components like files, databases, caches, and even external services. whatever you do will cause a chain reaction in the entire ecosystem if the other services aren't prepared, so there's no general answer to this. The general frame of mind would be: can your app do anything without the database? like validating JWT tokens, serving pages directly from redis, etc. If the database goes down,  is your app is worthless? I used the database example, but you should think on checking a route that will give you a reliable response time and not a false positive.
Answers: Magus
Answers: CC BY-SA 4.0
Answers: 2021-11-15T02:07:25.683Z
Answers: 2021-11-15T02:07:25.683Z
Answers: Magus
Body: <p>I have some doubts about the HTTP readinessProbe:</p>
  <ol>
  <li>Suppose that the probe fails several times (failureThreshold) and a pod is marked as NOT READY. Will k8s keep trying the readinessProbe forever and eventually mark the pod as Ready if it begins to respond correctly? Or a pod that is marked as Not Ready is left there forever without retrying?</li>
  <li>In case the Not Ready pod is left there forever, will the K8s Deployment try to start new pods? Or I will be left with less pods available?</li>
  <li>If a pod passes the readinessProbe, will k8s keep checking if it is ready and eventually (if it begins to fail) mark it as not ready in the future? Is that possible? Or a pod that is marked once as ready is ready forever?</li>
  </ol>
  
Tags: <kubernetes>
Title: Kubernetes readinessProbe: do health-checks run forever?
UserDisplayName: collimarco
ThreadText: # Kubernetes readinessProbe: do health-checks run forever?
  
  (Asked by: username_0 on 2021-11-08)
  
  I have some doubts about the HTTP readinessProbe:
  
  1. Suppose that the probe fails several times (failureThreshold) and a pod is marked as NOT READY. Will k8s keep trying the readinessProbe forever and eventually mark the pod as Ready if it begins to respond correctly? Or a pod that is marked as Not Ready is left there forever without retrying?
  2. In case the Not Ready pod is left there forever, will the K8s Deployment try to start new pods? Or I will be left with less pods available?
  3. If a pod passes the readinessProbe, will k8s keep checking if it is ready and eventually (if it begins to fail) mark it as not ready in the future? Is that possible? Or a pod that is marked once as ready is ready forever?
  
  **Tags:** kubernetes
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-11-15. Score: 2 | Accepted answer
  
  1 - yes, containers are left there forever. Think of readiness probe as a breaker, if the app gets overwhelmed it cuts the traffic to the pod until it's ready to get traffic again.
  
  2 - no, by itself, readiness probes only stop/start traffic to a pod. To signal kubernetes that the pod needs to be restarted use liveness probes.
  
  3 - yes, as per answer 1. Readiness probes are a way of letting kubernetes know that the pod can handle traffic or if they need a break.
  
  **probe: startup probes - waits for the app to start**
  
  use case: apps that need some time before start serving traffic, for example, building assets on start, pre-fetching assets, building caches, etc.
  
  **liveness probes - checks if the app is healthy**
  
  use case: check if the app is running, if the app has exited due to an exception and needs to be restarted, or if the app is running but can't serve requests
  
  **readiness probes - checks if the app can handle traffic**
  
  use case: apps that can arbitrarily stop receiving traffic, e.g. maintenance mode, data intensive apps that can degrade in performance and need a way to
  
  You should use liveness and readiness probes always, use startup probes when your app takes more time than usual to start.
  
  Example:
  
  * python app with postgres database:
  * liveness probe: get /health status 200
  * readiness probe: get /health status 200, timeout 1s
  
  the /health endpoint queries the database using a simple query (select id, email from user limit 1) and returns a json
  
  if the pod loses connectivity to the database, the liveness probe will fail and the pod will be restarted.
  if the database is being overwhelmed by requests, the query will take more than 1s, the probe will mark the pod as not ready, preventing that additional requests reach the database and preventing it from progressing from overwhelmed to crashed. once the database is able to catch up and starts returning queries in less than 1s, readiness probe will mark the pod as healthy again and kubernetes will resume serving requests to the pod.
  
  > **Comments:**
  > **username_0**: Thanks for the explanation! However I read that *liveness* probe shouldn't test a different component (like the database), because a failure in one component can damage the availability of entire application (with continuous restarts)  
  > **username_1**: yup! modern applications rely on several components like files, databases, caches, and even external services. whatever you do will cause a chain reaction in the entire ecosystem if the other services aren't prepared, so there's no general answer to this. The general frame of mind would be: can your app do anything without the database? like validating JWT tokens, serving pages directly from redis, etc. If the database goes down,  is your app is worthless? I used the database example, but you should think on checking a route that will give you a reliable response time and not a false positive.  
  
  ---
  
  