title: How to detect null column in pyspark
body: <p>I have a dataframe defined with some null values. Some Columns are fully null values.</p>
  
  
  
  <pre class="lang-python prettyprint-override"><code>&gt;&gt; df.show()
  +---+---+---+----+
  |  A|  B|  C|   D|
  +---+---+---+----+
  |1.0|4.0|7.0|null|
  |2.0|5.0|7.0|null|
  |3.0|6.0|5.0|null|
  +---+---+---+----+
  </code></pre>
  
  <p>In my case, I want to return a list of columns name that are filled with null values. My idea was to detect the constant columns (as the whole column contains the same null value).</p>
  
  <p>this is how I did it: </p>
  
  <pre class="lang-python prettyprint-override"><code>nullCoulumns = [c for c, const in df.select([(min(c) == max(c)).alias(c) for c in df.columns]).first().asDict().items() if const] 
  </code></pre>
  
  <p>but this does no consider null columns as constant, it works only with values.
  How should I then do it ?</p>