title: PySpark: match the values of a DataFrame column against another DataFrame column
body: <p>In Pandas DataFrame, I can use <code>DataFrame.isin()</code> function to match the column values against another column.</p>
  
  <p>For example: 
  suppose we have one DataFrame:</p>
  
  <pre><code>df_A = pd.DataFrame({'col1': ['A', 'B', 'C', 'B', 'C', 'D'], 
                       'col2': [1, 2, 3, 4, 5, 6]})
  df_A
  
      col1  col2
  0    A     1
  1    B     2
  2    C     3
  3    B     4
  4    C     5
  5    D     6         
  </code></pre>
  
  <p>and another DataFrame:</p>
  
  <pre><code>df_B = pd.DataFrame({'col1': ['C', 'E', 'D', 'C', 'F', 'G', 'H'], 
                       'col2': [10, 20, 30, 40, 50, 60, 70]})
  df_B
  
      col1  col2
  0    C    10
  1    E    20
  2    D    30
  3    C    40
  4    F    50
  5    G    60
  6    H    70       
  </code></pre>
  
  <p>I can use <code>.isin()</code> function to match the column values of <code>df_B</code> against the column values of <code>df_A</code></p>
  
  <p>E.g.:</p>
  
  <pre><code>df_B[df_B['col1'].isin(df_A['col1'])]
  </code></pre>
  
  <p>yields:</p>
  
  <pre><code>    col1  col2
  0    C    10
  2    D    30
  3    C    40
  </code></pre>
  
  <p><strong>What's the equivalent operation in PySpark DataFrame?</strong></p>
  
  <pre><code>df_A = pd.DataFrame({'col1': ['A', 'B', 'C', 'B', 'C', 'D'], 
                       'col2': [1, 2, 3, 4, 5, 6]})
  df_A = sqlContext.createDataFrame(df_A)
  
  df_B = pd.DataFrame({'col1': ['C', 'E', 'D', 'C', 'F', 'G', 'H'], 
                       'col2': [10, 20, 30, 40, 50, 60, 70]})
  df_B = sqlContext.createDataFrame(df_B)
  
  
  df_B[df_B['col1'].isin(df_A['col1'])]
  </code></pre>
  
  <p>The <code>.isin()</code> code above gives me an error messages:</p>
  
  <pre><code>u'resolved attribute(s) col1#9007 missing from 
  col1#9012,col2#9013L in operator !Filter col1#9012 IN 
  (col1#9007);;\n!Filter col1#9012 IN (col1#9007)\n+- 
  LogicalRDD [col1#9012, col2#9013L]\n'
  </code></pre>