Answers: <p>You have to login to that host where etcd pods is running (usually Mater node) and execute :</p>
  <pre><code>etcdctl defrag
  </code></pre>
  <p>OR</p>
  <pre><code>ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379  --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key defrag
  </code></pre>
  <p>prerequisite it to install etcdctl (by installing etcd to that host)</p>
  
Answers: CC BY-SA 4.0
Answers: 2023-07-20T11:13:59.747Z
Answers: 2023-07-20T11:13:59.747Z
Answers: Amandeep Singh
Answers: <p>You need to run the command on one of the Kubernetes nodes where the etcd pods are running.</p>
  <blockquote>
  <p>I'm running this from Git Bash. It errors out with a message that seems to make no sense. It's almost as though it's trying to use the local filesystem (which is a PC running Windows 11).</p>
  </blockquote>
  <p>That's because this is exactly what you are doing, and your Windows 11 install doesn't have the certificate files.</p>
  
Answers: CC BY-SA 4.0
Answers: 2023-05-23T08:45:47.897Z
Answers: 2023-05-23T08:45:47.897Z
Answers: guesty mcguest
Body: <p>I am trying to resolve a recurring Prometheus alert I'm getting that states that my Kubernetes cluster <code>etcd</code> database is fragmented and should be defragmented. I have never used the <code>etcdctl</code> command before, which is apparently installed automatically in the container in which <code>etcd</code> is running.</p>
  <p>I have found a <a href="https://blog.alekc.org/posts/fixing-etcddatabasehighfragmentationratio-prometheus-alert/" rel="nofollow noreferrer">post</a> that gives a very brief description of what to do, but it does not work for me. I have also been looking at documentation on the etcd.io <a href="https://etcd.io/docs/v3.5/op-guide/maintenance/#defragmentation" rel="nofollow noreferrer">site</a>, but everything there seems to assume the user is in the pod, running the <code>etcdctl</code> command directly.</p>
  <p>The command I have tried to use is:</p>
  <pre><code>kubectl exec $(kubectl get pods --selector=component=etcd -A -o name | head -n 1) -n kube-system -- etcdctl defrag --cluster --cacert /etc/kubernetes/pki/etcd/ca.crt --key /etc/kubernetes/pki/etcd/apiserver.key --cert /etc/kubernetes/pki/etcd/apiserver.crt
  </code></pre>
  <p>I am not 100% sure if I'm specifying the correct file paths for the <code>--key</code> and <code>--cert</code> options. The first link just uses <code>server.key</code> and <code>server.crt</code>, which it complains don't exist for my set-up. I did find the files <code>apiserver.key</code> and <code>apiserver.crt</code>, but maybe those are not the files to use. If they are not, I don't know what <strong>should</strong> be used.</p>
  <p>I'm running this from Git Bash. It errors out with a message that seems to make no sense. It's almost as though it's trying to use the local filesystem (which is a PC running Windows 11).</p>
  <p>The error message:</p>
  <pre><code>Error: open C:/Program Files/Git/etc/kubernetes/pki/etcd/apiserver.crt: no such file or directory
  command terminated with exit code 128
  </code></pre>
  <p>Are these directions correct? Does anyone know why I might be getting the strange error?</p>
  <p><strong>UPADTE</strong>:</p>
  <p>As suggested by @Botje, I ran the following (note the extra slash at the start of each file path):</p>
  <pre><code>$ kubectl exec $(kubectl get pods --selector=component=etcd -A -o name | head -n 1) -n kube-system -- etcdctl defrag --cluster --cacert //etc/kubernetes/pki/etcd/ca.crt --key //etc/kubernetes/pki/etcd/apiserver.key --cert //etc/kubernetes/pki/etcd/apiserver.crt
  </code></pre>
  <p>Unfortunately this did not work either:</p>
  <pre><code>Error: open //etc/kubernetes/pki/etcd/apiserver.crt: no such file or directory
  command terminated with exit code 128
  </code></pre>
  
Tags: <kubernetes><prometheus><etcd>
Title: How to defragment etcd database in Kubernetes cluster
UserDisplayName: Joseph Gagnon
ThreadText: # How to defragment etcd database in Kubernetes cluster
  
  (Asked by: username_0 on 2023-04-20)
  
  I am trying to resolve a recurring Prometheus alert I'm getting that states that my Kubernetes cluster `etcd` database is fragmented and should be defragmented. I have never used the `etcdctl` command before, which is apparently installed automatically in the container in which `etcd` is running.
  
  I have found a [post](https://blog.alekc.org/posts/fixing-etcddatabasehighfragmentationratio-prometheus-alert/) that gives a very brief description of what to do, but it does not work for me. I have also been looking at documentation on the etcd.io [site](https://etcd.io/docs/v3.5/op-guide/maintenance/#defragmentation), but everything there seems to assume the user is in the pod, running the `etcdctl` command directly.
  
  The command I have tried to use is:
  
  ```
  kubectl exec $(kubectl get pods --selector=component=etcd -A -o name | head -n 1) -n kube-system -- etcdctl defrag --cluster --cacert /etc/kubernetes/pki/etcd/ca.crt --key /etc/kubernetes/pki/etcd/apiserver.key --cert /etc/kubernetes/pki/etcd/apiserver.crt
  
  ```
  
  I am not 100% sure if I'm specifying the correct file paths for the `--key` and `--cert` options. The first link just uses `server.key` and `server.crt`, which it complains don't exist for my set-up. I did find the files `apiserver.key` and `apiserver.crt`, but maybe those are not the files to use. If they are not, I don't know what **should** be used.
  
  I'm running this from Git Bash. It errors out with a message that seems to make no sense. It's almost as though it's trying to use the local filesystem (which is a PC running Windows 11).
  
  The error message:
  
  ```
  Error: open C:/Program Files/Git/etc/kubernetes/pki/etcd/apiserver.crt: no such file or directory
  command terminated with exit code 128
  
  ```
  
  Are these directions correct? Does anyone know why I might be getting the strange error?
  
  **UPADTE**:
  
  As suggested by @Botje, I ran the following (note the extra slash at the start of each file path):
  
  ```
  $ kubectl exec $(kubectl get pods --selector=component=etcd -A -o name | head -n 1) -n kube-system -- etcdctl defrag --cluster --cacert //etc/kubernetes/pki/etcd/ca.crt --key //etc/kubernetes/pki/etcd/apiserver.key --cert //etc/kubernetes/pki/etcd/apiserver.crt
  
  ```
  
  Unfortunately this did not work either:
  
  ```
  Error: open //etc/kubernetes/pki/etcd/apiserver.crt: no such file or directory
  command terminated with exit code 128
  
  ```
  
  **Tags:** kubernetes, prometheus, etcd
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2023-07-20. Score: 1 | Accepted answer
  
  You have to login to that host where etcd pods is running (usually Mater node) and execute :
  
  ```
  etcdctl defrag
  
  ```
  
  OR
  
  ```
  ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379  --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key defrag
  
  ```
  
  prerequisite it to install etcdctl (by installing etcd to that host)
  
  
  ---
  
  ### Answer by username_2 on 2023-05-23. Score: 1
  
  You need to run the command on one of the Kubernetes nodes where the etcd pods are running.
  
  > I'm running this from Git Bash. It errors out with a message that seems to make no sense. It's almost as though it's trying to use the local filesystem (which is a PC running Windows 11).
  
  That's because this is exactly what you are doing, and your Windows 11 install doesn't have the certificate files.
  
  
  ---
  
  