title: Spark: Explode a dataframe array of structs and append id
body: <p>I currently have a dataframe with an id and a column which is <strong>an array of structs</strong>:</p>
  
  <pre><code> root
   |-- id: integer (nullable = true)
   |-- lists: array (nullable = true)
   |    |-- element: struct (containsNull = true)
   |    |    |-- text: string (nullable = true)
   |    |    |-- amount: double (nullable = true)
  </code></pre>
  
  <p>Here is an example table with data:</p>
  
  <pre><code> id | lists
   -----------
   1  | [[a, 1.0], [b, 2.0]]
   2  | [[c, 3.0]]
  </code></pre>
  
  <p>How do I transform the above dataframe to the one below? I need to "explode" the array and append the id at the same time.</p>
  
  <pre><code> id | col1  | col2
   -----------------
   1  | a     | 1.0
   1  | b     | 2.0
   2  | c     | 3.0
  </code></pre>
  
  <hr>
  
  <p><strong>Edited Note:</strong></p>
  
  <p>Note there is a difference between the two examples below. The first one contains <strong>"an array of structs of elements"</strong>.  While the later just contains <strong>"an array of elements"</strong>.</p>
  
  <pre><code> root
   |-- id: integer (nullable = true)
   |-- lists: array (nullable = true)
   |    |-- element: struct (containsNull = true)
   |    |    |-- text: string (nullable = true)
   |    |    |-- amount: double (nullable = true)
  
  
  root
   |-- a: long (nullable = true)
   |-- b: array (nullable = true)
   |    |-- element: long (containsNull = true)
  </code></pre>