title: Which is best in Python: urllib2, PycURL or mechanize?
body: <p>Ok so I need to download some web pages using Python and did a quick investigation of my options.</p>
  
  <p>Included with Python:</p>
  
  <p><a href="http://docs.python.org/library/urllib.html" rel="noreferrer">urllib</a> - seems to me that I should use urllib2 instead. urllib has no cookie support, HTTP/FTP/local files only (no SSL)</p>
  
  <p><a href="http://docs.python.org/library/urllib2.html" rel="noreferrer">urllib2</a> - complete HTTP/FTP client, supports most needed things like cookies, does not support all HTTP verbs (only GET and POST, no TRACE, etc.)</p>
  
  <p>Full featured:</p>
  
  <p><a href="http://wwwsearch.sourceforge.net/mechanize/" rel="noreferrer">mechanize</a> - can use/save Firefox/IE cookies, take actions like follow second link, actively maintained (0.2.5 released in March 2011)</p>
  
  <p><a href="http://pycurl.sourceforge.net/" rel="noreferrer">PycURL</a> - supports everything curl does (FTP, FTPS, HTTP, HTTPS, GOPHER, TELNET, DICT, FILE and LDAP), bad news: not updated since Sep 9, 2008 (7.19.0)</p>
  
  <p>New possibilities:</p>
  
  <p><a href="http://code.google.com/p/urllib3/" rel="noreferrer">urllib3</a> - supports connection re-using/pooling and file posting</p>
  
  <p>Deprecated (a.k.a. use urllib/urllib2 instead):</p>
  
  <p><a href="http://docs.python.org/library/httplib.html" rel="noreferrer">httplib</a> - HTTP/HTTPS only (no FTP)</p>
  
  <p><a href="http://code.google.com/p/httplib2/" rel="noreferrer">httplib2</a> - HTTP/HTTPS only (no FTP)</p>
  
  <p>The first thing that strikes me is that urllib/urllib2/PycURL/mechanize are all pretty mature solutions that work well. mechanize and PycURL ship with a number of Linux distributions (e.g. Fedora 13) and BSDs so installation is a non issue typically (so that's good). </p>
  
  <p>urllib2 looks good but I'm wondering why PycURL and mechanize both seem very popular, is there something I am missing (i.e. if I use urllib2 will I paint myself in to a corner at some point?). I'd really like some feedback on the pros/cons of these things so I can make the best choice for myself. </p>
  
  <p>Edit: added note on verb support in urllib2</p>