title: Is there a way to add a column of type dictionary to a spark dataframe in pyspark?
body: <p>This is how I create a dataframe with primitive data types in <code>pyspark</code>:</p>
  
  <pre><code>from pyspark.sql.types import StructType, StructField, DoubleType, StringType, IntegerType
  fields = [StructField('column1', IntegerType(), True), StructField('column2', IntegerType(), True)]
  schema = StructType(fields)
  df = spark.createDataFrame([], schema)
  values = [tuple([i]) +
            tuple([i])
            for i in range(3)]
  df = spark.createDataFrame(values, schema)
  </code></pre>
  
  <p>Now, if I want to have a third column with dictionary data, eg: {"1": 1.0, "2": 2.0, "3": 3.0}, what should I do?
  I want to create this data frame:</p>
  
  <pre><code>+--------------------+-----------------+------------------------------+
  |column1             |column2          |column3                       |
  +--------------------+-----------------+------------------------------+
  |1                   |1                |{"1": 1.0, "2": 1.0, "3": 1.0}|
  +--------------------+-----------------+------------------------------+
  |2                   |2                |{"1": 2.0, "2": 2.0, "3": 2.0}|
  +--------------------+-----------------+------------------------------+
  |3                   |3                |{"1": 3.0, "2": 3.0, "3": 3.0}|
  +--------------------+-----------------+------------------------------+
  </code></pre>
  
  <p>There is a MapType that seems to be helpful, but I can't figure out how to use it? </p>
  
  <p>And assuming the data frame is created, how to filter it based on the third column, given a dict to select the rows of the data frame that have that dict value?</p>