How does xpathselector affect the speed of scrapy crawl running?
I am afraid that you might look for 'something' in the entire document, so you probably should still use hxs.select('//*').re('something').
And about the speed question: the answer is that if you look for the word 'something' in a document which is 4k large, of course it will take longer then filtering the document for text() and after looking for that word within that text.
I am using Scrapy crawler to crawl over 100k pages website.  The speed is the big concern in this case.  Today I noticed that hxs.select('//*').re('something') is way slower than hxs.select('//script/text()').re('something').  Can any expert explain to me why? 
As I understand, the crawler should download the entire page no matter what xpath selector I use.  So the xpath should not affect the speed much at all.
Thanks a lot for any tips.