title: PySpark: filtering a DataFrame by date field in range where date is string
body: <p>My dataframes contains one field which is a date and it appears in the string format, as example</p>
  
  <pre><code>'2015-07-02T11:22:21.050Z'
  </code></pre>
  
  <p>I need to filter the DataFrame on the date to get only the records in the last week. 
  So, I was trying a map approach where I transformed the string dates to datetime objects with strptime: </p>
  
  <pre><code>def map_to_datetime(row):
       format_string = '%Y-%m-%dT%H:%M:%S.%fZ'
       row.date = datetime.strptime(row.date, format_string)
  
  df = df.map(map_to_datetime)
  </code></pre>
  
  <p>and then I would apply a filter as </p>
  
  <pre><code>df.filter(lambda row:
      row.date &gt;= (datetime.today() - timedelta(days=7)))
  </code></pre>
  
  <p>I manage to get the mapping working but the filter fails with </p>
  
  <blockquote>
    <p>TypeError: condition should be string or Column</p>
  </blockquote>
  
  <p>Is there a way to use a filtering in a way that works or should I change the approach and how?</p>