title: Comparing schema of dataframe using Pyspark
body: <p>I have a data frame (df).
  For showing its schema I use:</p>
  
  <pre><code>from pyspark.sql.functions import *
  df1.printSchema()
  </code></pre>
  
  <p>And I get the following result:</p>
  
  <pre><code>#root
  # |-- name: string (nullable = true)
  # |-- age: long (nullable = true)
  </code></pre>
  
  <p>Sometimes the schema changes (the column type or name): </p>
  
  <pre><code>df2.printSchema()
  
  
   #root
          # |-- name: array (nullable = true)
          # |-- gender: integer (nullable = true)
          # |-- age: long (nullable = true)
  </code></pre>
  
  <p>I would like to compare between the two schemas (<code>df1</code> and <code>df2</code>) and get only the differences in types and columns names (Sometimes the column can move to another position). 
  The results should be a table (or data frame) something like this:</p>
  
  <pre><code>   column                df1          df2     diff                       
      name:               string       array     type                             
      gender:              N/A         integer   new column 
  </code></pre>
  
  <p>(<code>age</code> column is the same and didn't change. In case of omission of column there will be indication <code>'omitted'</code>)
  How can I do it if efficiently if I have many columns in each?</p>