I have to wonder if there have been some recent changes which are causing all kinds of issues.  Just a few days ago, Ryan Tomayko twittered about it completely ignoring Github's robots.txt - <a href="http://twitter.com/rtomayko/statuses/7685967826" rel="nofollow">http://twitter.com/rtomayko/statuses/7685967826</a>