title: How do I interpret Input size / records in Spark Stage UI
body: <p>I'm looking at the Spark UI (Spark v1.6.0) for a stage of a job I'm currently running and I don't understand how to interpret what its telling me:
  <a href="https://i.stack.imgur.com/Sy3Vv.png" rel="noreferrer"><img src="https://i.stack.imgur.com/Sy3Vv.png" alt="Spark stage UI"></a>
  The number of records in the "Shuffle Write Size / Records" column makes sense, those numbers are consistent with the data I'm processing.</p>
  
  <p>What I do not understand is the numbers in "Input Size / Records". They indicate that the incoming data has only ~67 records in each partition; the job has 200 partitions so ~1200 records in all. I dont know what that is referring to, none of the input datasets to this job (which was implemented using SparkSQL) have ~1200 records in them.</p>
  
  <p>So, I'm flummoxed as to what those numbers are referring to. Can anyone enlighten me?</p>