title: Spark: Convert column of string to an array
body: <p>How to convert a column that has been read as a string into a column of arrays?
  i.e. convert from below schema</p>
  
  <pre><code>scala&gt; test.printSchema
  root
   |-- a: long (nullable = true)
   |-- b: string (nullable = true)
  
  +---+---+
  |  a|  b|
  +---+---+
  |  1|2,3|
  +---+---+
  |  2|4,5|
  +---+---+
  </code></pre>
  
  <p>To: </p>
  
  <pre><code>scala&gt; test1.printSchema
  root
   |-- a: long (nullable = true)
   |-- b: array (nullable = true)
   |    |-- element: long (containsNull = true)
  
  +---+-----+
  |  a|  b  |
  +---+-----+
  |  1|[2,3]|
  +---+-----+
  |  2|[4,5]|
  +---+-----+
  </code></pre>
  
  <p>Please share both scala and python implementation if possible. 
  On a related note, how do I take care of it while reading from the file itself?
  I have data with ~450 columns and few of them I want to specify in this format.
  Currently I am reading in pyspark as below:</p>
  
  <pre><code>df = spark.read.format('com.databricks.spark.csv').options(
      header='true', inferschema='true', delimiter='|').load(input_file)
  </code></pre>
  
  <p>Thanks.</p>