Spambots are cluttering my log file [Django]
How about setting up a catch-all pattern as the last item in your urls file and directing it to a generic "no such page" or even your homepage? In other words, turn 500's into requests for your homepage.
I have a nice and lovely Django site up and running, but have noticed that my error.log file was getting huge, over 150 MB after a couple of months of being live. Turns out a bunch of spambots are looking for well known URL vulnerabilities (or something) and hitting a bunch of sub-directories like http://mysite.com/ie or http://mysite.com/~admin.php etc. 
Since Django uses URL rewriting, it is looking for templates to fit these requests, which raises a TemplateDoesNotExist exception, and then a 500 message (Django does this, not me). I have debug turned off, so they only get the generic 500 message, but it's filling up my logs very quickly.
Is there a way to turn this behavior off? Or perhaps just block the IP's doing this?