title: How to crawl a web page for files of certain size
body: <p>I need to crawl a list of several thousand hosts and find at least two files rooted there that are larger than some value, given as an argument. Can any popular (python based?) tool possibly help?</p>