title: Kafka Error Handling using Shopify Sarama
body: <p>So I am trying to use Kafka for my application which has a producer logging actions into the Kafka MQ and the consumer which reads it off the MQ.Since my application is in Go, I am using the Shopify Sarama to make this possible.</p>
  
  <p>Right now, I'm able to read off the MQ and print the message contents using a</p>
  
  <pre><code>fmt.Printf
  </code></pre>
  
  <p>Howeveer, I would really like the error handling to be better than console printing and I am willing to go the extra mile. </p>
  
  <p>Code right now for consumer connection:</p>
  
  <pre><code>mqCfg := sarama.NewConfig()
  
  master, err := sarama.NewConsumer([]string{brokerConnect}, mqCfg)
  if err != nil {
      panic(err) // Don't want to panic when error occurs, instead handle it
  }
  </code></pre>
  
  <p>And the processing of messages:</p>
  
  <pre><code>    go func() {
      defer wg.Done()
      for message := range consumer.Messages() {
          var msgContent Message
          _ = json.Unmarshal(message.Value, &amp;msgContent)
          fmt.Printf("Reading message of type %s with id : %d\n", msgContent.Type, msgContent.ContentId) //Don't want to print it
      }
  }()
  </code></pre>
  
  <p>My questions (I am new to testing Kafka and new to kafka in general):</p>
  
  <ol>
  <li><p>Where could errors occur in the above program, so that I can handle them? Any sample code will be great for me to start with. The error conditions I could think of are when the msgContent doesn't really contain any Type of ContentId fields in the JSON. </p></li>
  <li><p>In kafka, are there situations when the consumer is trying to read at the current offset, but for some reason was not able to (even when the JSON is well formed)? Is it possible for my consumer to backtrack to say x steps above the failed offset read and re-process the offsets? Or is there a better way to do this? again, what could these situations be?</p></li>
  </ol>
  
  <p>I'm open to reading and trying things. </p>