title: Finding email addresses in body using scrapy
body: <p>I am trying to find all the email addresses on a page using scrapy.</p>
  
  <p>I found a xpath which should return the email addresses but when I run the code below it doesnt find any email addresses (which I know are there). And I get errors like:</p>
  
  <p>File "C:\Anaconda2\lib\site-packages\scrapy\selector\unified.py", line 100, in
   xpath
      raise ValueError(msg if six.PY3 else msg.encode("unicode_escape"))
  ValueError: Invalid XPath: //[-a-zA-Z0-9.<em>]+@[-a-zA-Z0-9</em>]+.[a-zA-Z0-9_.]+</p>
  
  <p>This is what my code looks like. Can someone tell me what I'm doing wrong?</p>
  
  <p>I've narrowed down the problem to the xpath but cannot figure out how to fix it.</p>
  
  <pre><code>import scrapy
  import datetime
  from scrapy.spiders import CrawlSpider
  from techfinder.items import EmailItem
  from scrapy.selector import HtmlXPathSelector
  
  
  class DetectSpider(scrapy.Spider):
      name = "test"
  
      alloweddomainfile = open("emaildomains.txt")
      allowed_domains = [domain.strip() for domain in alloweddomainfile.readlines()]
      alloweddomainfile.close()
  
      starturlfile = open("emailurls.txt")
      start_urls = [url.strip() for url in starturlfile.readlines()]
      starturlfile.close()
  
  
      def parse(self, response):
  
  
  
  
          hxs = HtmlXPathSelector(response)
  
  
          emails = hxs.xpath('//[-a-zA-Z0-9._]+@[-a-zA-Z0-9_]+.[a-zA-Z0-9_.]+').extract()             
          #[-a-zA-Z0-9._]+@[-a-zA-Z0-9_]+.[a-zA-Z0-9_.]+
          #&lt;a\s+href=\"mailto:([a-zA-Z0-9._@]*)\
          #/^(|(([A-Za-z0-9]+_+)|([A-Za-z0-9]+\-+)|([A-Za-z0-9]+\.+)|([A-Za-z0-9]+\++))*[A-Za-z0-9]+@((\w+\-+)|(\w+\.))*\w{1,63}\.[a-zA-Z]{2,6})$/i
  
  
  
          emailitems = []
          for email in zip(emails):
              emailitem = EmailItem()
              emailitem["email"] = emails
              emailitem["source"] = response.url
              emailitem["datetime"] = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
              emailitems.append(emailitem)
          return emailitems
  </code></pre>