title: How to fix 'Kafka Offset commit failed on partition: The request timed out'
body: <p>I suddenly got exceptions of type in production Kafka</p>
  
  <pre class="lang-java prettyprint-override"><code>ERROR[pool-XX-thread-YY] org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=someclientid, groupId=somegroup] Offset commit failed on partition SomeTopic-SomePartition at offset SomeOffset: The request timed out.
  </code></pre>
  
  <p>This occurred for 3.5 seconds from a lot of different services (clients) (different threads\different topics\different partitions) and than just self healed...
  The offset commit configurations is 5 second auto-commit for all those clients.</p>
  
  <p>Couldn't track anything from kafka broker logs except of some re-balancing right of one group (out of 10 had that issue) which is normal when heart-beating fails, in the metrics server I can see some spikes of commit latency which is the symptom I guess and some TCP spikes on 1 broker (out of 3)</p>
  
  <p>How can I start investigate it? what can cause such an issue? where should I look when things like that is occurring?</p>
  
  <p>Attaching photos of some graphs here:</p>
  
  <p><strong>TCP Spike in server-3</strong>
  <a href="https://i.stack.imgur.com/fuV2r.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/fuV2r.png" alt="TCP Spike in server-3"></a></p>
  
  <p><strong>Commit latency spike</strong>
  <a href="https://i.stack.imgur.com/8nFs8.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/8nFs8.png" alt="Commit latency spike"></a></p>
  
  <p><strong>Group syncs</strong>
  <a href="https://i.stack.imgur.com/glNZ5.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/glNZ5.png" alt="Group syncs"></a></p>
  
  <p><strong>Heartbeats</strong>
  <a href="https://i.stack.imgur.com/iS5ic.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/iS5ic.png" alt="Heartbeats"></a></p>