title: 100,000 page renders using headless browser, max per cpu per second on aws
body: <p>I'm doing a scraping project and need to use a browser to load javascript on websites that I visit. </p>
  
  <p>If using just http I can get up to <code>150/second</code> on an <code>m4.large</code>. Which has <code>2 cpus</code>. On initial tests using headless chrome its so much slower around <code>4 / second</code> and typically the bottleneck is CPU not RAM. For many web pages basic http is fine but increasingly its not with single page apps etc.</p>
  
  <p>Does anyone have some experience doing something similar? What were you able to achieve? What was the stack you used e.g. <code>python, selenium, headless chrome and multiprocessing</code>. Did you have to alter the configuration of chrome?</p>