title: kafka: replicas and ISR don't match
body: <p>Im moving hundreds of topics from one broker to another. The process is</p>
  
  <ol>
  <li>Use <code>kafka-topics.sh</code> to generate existing partition list</li>
  <li>Use <code>kafka-reassign-partitions.sh</code> to generate current list of partitions / brokers / etc</li>
  <li>Edit this list so every instance of <strong>broker 7(to be replaced)</strong> is now <strong>brokers 7,4</strong> (4 is new broker)</li>
  <li>Run <code>kafka-reassign-partitions.sh (broker list) --execute</code> to add new broker</li>
  <li>Wait.. watch.. (using --verify).. until complete...</li>
  <li>Edit broker list to be <strong>brokers 4,7</strong></li>
  <li>Do (4) again.. do (5) again.. </li>
  <li>Run preferred leader election (in case 7 was leading anything)</li>
  <li>Edit broker list to remove all instances of <strong>broker 7</strong></li>
  <li>Do (4) and (5) again</li>
  <li>Be happy</li>
  </ol>
  
  <p>This has worked great for hundreds and hundreds of topics.. except for one sticky one. This holdout is refusing to sync up with the new broker (missing from ISR list) even though it's included in list of replicas</p>
  
  <p>Output from <code>kafka-topics.sh</code> (trying to replace broker 7 with broker 4):</p>
  
  <blockquote>
    <p>Topic: shard_3  Partition: 7    Leader: 3   Replicas: 3,4,7 Isr: 7,3</p>
  </blockquote>
  
  <p>I've run (4) above several times in hopes of getting this to complete but it doesn't seem to want to. I've waited overnight in case it's just really slow.</p>
  
  <p>Suggestions on how to unstick this one?</p>