title: RUBY - web scraping - (OpenURI::HTTPError)
body: <p>I'm trying to code a simple web scraping in ruby.
  It works till 29th url then I get this error message:</p>
  
  <blockquote>
  <pre><code>C:/Ruby193/lib/ruby/1.9.1/open-uri.rb:346:in `open_http': 500 Internal Server Er
  ror (OpenURI::HTTPError)
          from C:/Ruby193/lib/ruby/1.9.1/open-uri.rb:775:in `buffer_open'
          from C:/Ruby193/lib/ruby/1.9.1/open-uri.rb:203:in `block in open_loop'
          from C:/Ruby193/lib/ruby/1.9.1/open-uri.rb:201:in `catch'
          from C:/Ruby193/lib/ruby/1.9.1/open-uri.rb:201:in `open_loop'
          from C:/Ruby193/lib/ruby/1.9.1/open-uri.rb:146:in `open_uri'
          from C:/Ruby193/lib/ruby/1.9.1/open-uri.rb:677:in `open'
          from C:/Ruby193/lib/ruby/1.9.1/open-uri.rb:33:in `open'
          from test.rb:24:in `block (2 levels) in &lt;main&gt;'
          from test.rb:18:in `each'
          from test.rb:18:in `block in &lt;main&gt;'
          from test.rb:14:in `each'
          from test.rb:14:in `&lt;main&gt;'
  </code></pre>
  </blockquote>
  
  <p>My code:</p>
  
  <pre><code>require 'rubygems'  
  require 'nokogiri'  
  require 'open-uri'  
  
  aFile=File.new('data.txt', 'w')
  
  ag = 0
    for i in 1..40 do
      agenzie = ag + 1
  
      #change url parameter 
  
      url = "http://www.infotrav.it/dettaglio.do?sort=*RICOVIAGGI*&amp;codAgenzia=" + "#{ ag }"  
      doc = Nokogiri::HTML(open(url))
      aFile=File.open('data.txt', 'a')
      aFile.write(doc.at_css("table").text)
      aFile.close
    end
  </code></pre>
  
  <p>Do you have some ideas to solve it?
  Thanks!</p>
  
  <p>aS</p>