title: Filter PySpark DataFrame by checking if string appears in column
body: <p>I'm new to Spark and playing around with filtering. I have a pyspark.sql DataFrame created by reading in a json file. A part of the schema is shown below:</p>
  
  <pre><code>root
   |-- authors: array (nullable = true)
   |    |-- element: string (containsNull = true)
  </code></pre>
  
  <p>I would like to filter this DataFrame, selecting all of the rows with entries pertaining to a particular author. So whether this author is the first author listed in <code>authors</code> or the nth, the row should be included if their name appears. So something along the lines of</p>
  
  <pre><code>df.filter(df['authors'].getItem(i)=='Some Author')
  </code></pre>
  
  <p>where <code>i</code> iterates through all authors in that row, which is not constant across rows.</p>
  
  <p>I tried implementing the solution given to <a href="https://stackoverflow.com/questions/38019921/pyspark-dataframes-filter-where-some-value-is-in-array-column">PySpark DataFrames: filter where some value is in array column</a>, but it gives me</p>
  
  <blockquote>
    <p><strong>ValueError:</strong> Some of types cannot be determined by the first 100 rows,
    please try again with sampling</p>
  </blockquote>
  
  <p>Is there a succinct way to implement this filter?</p>