title: MS SQL CDC with Kafka Connect and Apache Kafka
body: <p>In my current use case, I am using Spark core to read data from MS SQL Server and doing some processing on the data and sending it to Kafka every 1 minute, I am using Spark and Phoenix to maintain the CDC information in HBase table.</p>
  
  <p>But this design has some issues, for e.g. if there is surge in MS SQL records Spark processing takes more time than batch interval and spark ends up sending duplicate records to Kafka.</p>
  
  <p>As an alternate to this I am thinking of using Kafka Connect to read  the messages from MS SQL and send records to Kafka topic and maintain the MS SQL CDC in Kafka. Spark Streaming will read records from Kafka topic and process the records and stores into HBase and send to other Kafka topics.</p>
  
  <p>I have a few questions in order to implement this architecture:</p>
  
  <ol>
  <li><p>Can I achieve this architecture with open source Kafka connectors and Apache Kafka 0.9 versions.</p></li>
  <li><p>If yes can you please recommend me a GitHub project, which can offer me such connectors where I can CDC MS SQL tables using SQL query such as <code>SELECT * FROM SOMETHING WHERE COLUMN &gt; ${lastExtractUnixTime})</code> and store records into Kafka topic.</p></li>
  <li><p>Does Kafka connect supports Kerberos Kafka setup.</p></li>
  </ol>