title: My Scrapes terminated once at 100 records, then 66, 50, 50! Can I slow down my Scraping some how?
body: <p>I am very new to screen scraping and would like to know if someone can tell me how to slow the scraping process so that my queries tend to be more human like and not an obvious bot. </p>
  
  <p>Here's what I have for my curl_setopt:</p>
  
  <pre><code>curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 180);
  
  curl_setopt($ch, CURLOPT_TIMEOUT, 300);
  
  
  curl_setopt($ch,CURLOPT_USERAGENT,'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13');
  
  
  curl_setopt ($this-&gt;ch, CURLOPT_RETURNTRANSFER, 1);
  
  
  curl_setopt ($this-&gt;ch, CURLOPT_URL, $this-&gt;url); 
  
  
  curl_setopt($this-&gt;ch, CURLOPT_FOLLOWLOCATION, true); 
  
  
  curl_setopt($this-&gt;ch, CURLOPT_BINARYTRANSFER, $this-&gt;binary); 
  </code></pre>