title: How to spider a password protected site in python?
body: <p>currently I have a spider written in Java that logs into a supplier website and spiders the website. (using htmlunit)</p>
  
  <p>It keeps the session (cookie) and even lets me enable/disable javascript etc.</p>
  
  <p>I also use htmlparser (java) to help parse the html and extract the relevant information.</p>
  
  <p>Does python have something similar to do this?</p>