title: How to partition RDD by key in Spark?
body: <p>Given that the HashPartitioner docs say:</p>
  
  <blockquote>
    <p>[HashPartitioner] implements hash-based partitioning using Java's
    Object.hashCode.</p>
  </blockquote>
  
  <p>Say I want to partition <code>DeviceData</code> by its <code>kind</code>.</p>
  
  <pre><code>case class DeviceData(kind: String, time: Long, data: String)
  </code></pre>
  
  <p>Would it be correct to partition an <code>RDD[DeviceData]</code> by overwriting the <code>deviceData.hashCode()</code> method and use only the hashcode of <code>kind</code>?</p>
  
  <p>But given that <code>HashPartitioner</code> takes a number of partitions parameter I am confused as to whether I need to know the number of kinds in advance and what happens if there are more kinds than partitions?</p>
  
  <p>Is it correct that if I write partitioned data to disk it will stay partitioned when read?</p>
  
  <p>My goal is to call </p>
  
  <pre><code>  deviceDataRdd.foreachPartition(d: Iterator[DeviceData] =&gt; ...)
  </code></pre>
  
  <p>And have only <code>DeviceData</code>'s of the same <code>kind</code> value in the iterator.</p>