title: Spark collect_list change data_type from array to string
body: <p>I am having a following aggregation</p>
  <pre class="lang-scala prettyprint-override"><code>val df_date_agg = df
      .groupBy($&quot;a&quot;,$&quot;b&quot;,$&quot;c&quot;)
      .agg(sum($&quot;d&quot;).alias(&quot;data1&quot;),sum($&quot;e&quot;).alias(&quot;data2&quot;))
      .groupBy($&quot;a&quot;)
      .agg(collect_list(array($&quot;b&quot;,$&quot;c&quot;,$&quot;data1&quot;)).alias(&quot;final_data1&quot;),
           collect_list(array($&quot;b&quot;,$&quot;c&quot;,$&quot;data2&quot;)).alias(&quot;final_data2&quot;))
  </code></pre>
  <p>Here I am doing some aggregation and collecting the result with <code>collect_list</code>. Earlier we were using spark 1 and it was giving me below data types.</p>
  <pre><code> |-- final_data1: array (nullable = true)
   |    |-- element: string (containsNull = true)
   |-- final_data2: array (nullable = true)
   |    |-- element: string (containsNull = true)
  </code></pre>
  <p>Now we have to migrate to spark 2 but we are getting below schema.</p>
  <pre><code>|-- final_data1: array (nullable = true)
   |    |-- element: array (containsNull = true)
   |    |    |-- element: string (containsNull = true)
   |-- final_data1: array (nullable = true)
   |    |-- element: array (containsNull = true)
   |    |    |-- element: string (containsNull = true)
  </code></pre>
  <p>On getting <code>first()</code> record below is the difference</p>
  <pre><code>spark 1.6
  
  [2020-09-26, Ayush, 103.67] =&gt; datatype string
  
  spark 2 
  
  WrappedArray(2020-09-26, Ayush, 103.67)
  </code></pre>
  <p>How can I keep the same data type?</p>
  <p><strong>Edit - Tried Using Concat</strong></p>
  <p>One way I got exact schema like Spark 1.6 is by using concat like this</p>
  <pre><code>val df_date_agg = df
      .groupBy($&quot;msisdn&quot;,$&quot;event_date&quot;,$&quot;network&quot;)
      .agg(sum($&quot;data_mou&quot;).alias(&quot;data_mou_dly&quot;),sum($&quot;voice_mou&quot;).alias(&quot;voice_mou_dly&quot;))
      .groupBy($&quot;msisdn&quot;)
      .agg(collect_list(concat(lit(&quot;[&quot;),lit($&quot;event_date&quot;),lit(&quot;,&quot;),lit($&quot;network&quot;),lit(&quot;,&quot;),lit($&quot;data_mou_dly&quot;),lit(&quot;]&quot;)))
  </code></pre>
  <p>Will it affect my code performance?? Is there a better way to do this?</p>