I wrote a web crawler to crawl product infomation from www.amazon.com by using urllib2,but it seems that amazon limit the connection for each IP to 1. 
When I start more than one thread to crawl simultaneously, it raises HTTP Error 503: Service Temporarily Unavailable.
I want to start more threads to crawl fast,so how can I fix this error?
You should probably switch to use the Amazon API for product queries.
how to crawl web pages fast when the number of connection is limited