title: Does Spark do UnionAll in parallel?
body: <p>I got 10 <code>DataFrame</code>s with the same schema which I'd like to combine into one <code>DataFrame</code>. Each <code>DataFrame</code> is constructed using a <code>sqlContext.sql("select ... from ...").cahce</code>, which means that technically, the <code>DataFrame</code>s are not really calculated until it's time to use them.</p>
  
  <p>So, if I run:</p>
  
  <pre><code>val df_final = df1.unionAll(df2).unionAll(df3).unionAll(df4) ...
  </code></pre>
  
  <p>will Spark calculate all these <code>DataFrame</code>s in parallel or one by one (due to the dot operator)?</p>
  
  <p>And also, while we're here - is there a more elegant way to preform a <code>unionAll</code> on several <code>DataFrame</code>s than the one I listed above?</p>