title: An alternative web crawler to Nutch
body: <p>I'm trying to build a specialised search engine web site that indexes a limited number of web sites. The solution I came up with is:</p>
  
  <ul>
  <li>using Nutch as the web crawler,</li>
  <li>using Solr as the search engine,</li>
  <li>the front-end and the site logic is coded with Wicket.</li>
  </ul>
  
  <p>The problem is that I find Nutch quite complex and it's a big piece of software to customise, despite the fact that a detailed documentation (books, recent tutorials.. etc) does just not exist.</p>
  
  <p>Questions now:</p>
  
  <ol>
  <li>Any constructive criticism about the hole idea of the site?</li>
  <li>Is there a good yet simple alternative to Nutch (as the crawling part of the site)?</li>
  </ol>
  
  <p>Thanks </p>