I think they might mitigate the need to crawl _every_ page of every web site in that fashion. They must be doing some sort of analysis to &quot;old-school-crawl&quot; pages that don&#x27;t need javascript interpretation.