title: Import text files to pig through python UDF
body: <p>I'm trying to load files to pig while use python udf, i've tried two ways:</p>
  
  <p>•   (myudf1, sample1.pig): try to read the file from python, the file is located on my client server.</p>
  
  <p>•   (myudf2, sample2.pig): load file from hdfs to grunt shell first, then pass it as a parameter to python udf.</p>
  
  <p>myudf1.py</p>
  
  <pre><code>from __future__ import with_statement
  def get_words(dir):
      stopwords=set()
      with open(dir) as f1:
          for line1 in f1:
              stopwords.update([line1.decode('ascii','ignore').split("\n")[0]])
      return stopwords
  
  stopwords=get_words("/home/zhge/uwc/mappings/english_stop.txt")
  
  @outputSchema("findit: int")
  def findit(stp):
      stp=str(stp)
      if stp in stopwords:
          return 1
      else:
          return 0
  </code></pre>
  
  <p>sample1.pig:</p>
  
  <pre><code>REGISTER '/home/zhge/uwc/scripts/myudf1.py' USING jython as pyudf;
  item_title = load '/user/zhge/data/item_title_sample/000000_0' USING PigStorage(',')  AS (title:chararray);
  
  T = limit item_title 1;
  S = FOREACH T GENERATE pyudf.findit(title);
  DUMP S
  </code></pre>
  
  <p><strong>I get: IOError: (2, 'No such file or directory', '/home/zhge/uwc/mappings/english_stop.txt')</strong></p>
  
  <p>For solution 2:</p>
  
  <p>myudf2:</p>
  
  <pre><code>def get_wordlists(wordbag):
      stopwords=set()
      for t in wordbag:
          stopwords.update(t.decode('ascii','ignore'))
      return stopwords
  
  
  @outputSchema("findit: int")
  def findit(stopwordbag, stp):
      stopwords=get_wordlists(stopwordbag)
      stp=str(stp)
      if stp in stopwords:
          return 1
      else:
          return 0
  </code></pre>
  
  <p>Sample2.pig</p>
  
  <pre><code>REGISTER '/home/zhge/uwc/scripts/myudf2.py' USING jython as pyudf;
  
  stops = load '/user/zhge/uwc/mappings/stopwords.txt' AS (stop_w:chararray);
  -- this step works fine and i can see the "stops" obejct is loaded to pig 
  item_title = load '/user/zhge/data/item_title_sample/000000_0' USING PigStorage(',')  AS (title:chararray);
  T = limit item_title 1;
  S = FOREACH T GENERATE pyudf.findit(stops.stop_w, title);
  DUMP S;
  </code></pre>
  
  <p>Then I got: 
  <strong>ERROR org.apache.pig.tools.grunt.Grunt -ERROR 1066: Unable to open iterator for alias S. Backend error : Scalar has more than one row in the output. 1st : (a), 2nd :(as</strong></p>