title: Replacing empty string with null leads to INCREASE in dataframe size?
body: <p>I'm having trouble understanding the following phenomenon: in Spark 2.2, on Scala, I witness a significant incease in the persisted DataFrame size after replacing literal empty string values with lit(null).</p>
  
  <p>This is the function I use to replace empty string values:</p>
  
  <pre><code>def nullifyEmptyStrings(df:DataFrame): DataFrame = {
      var in = df
      for (e &lt;- df.columns) {
           in = in.withColumn(e, when(length(col(e))===0, lit(null:String)).otherwise(col(e)))
       }
      in
    }
  </code></pre>
  
  <p>I observe that the persisted (DISK_ONLY) size of my initial dataframe before running this function is 1480MB, and afterwards is 1610MB. The number of partitions remains unchanged.</p>
  
  <p>Any thoughts? The nulling works fine by the way, but my main reason for introducing this was to reduce shuffle size, and it seems I only increase it this way.</p>