Cloud server showdown: Amazon AWS EC2 vs. Linode vs. DigitalOcean
<i>sigh</i><p>I call these "weekend benchmarks" -- what you'd typically do when you have a block of free time, then spend time optimizing for said benchmark. Roll in on Monday with some staggering results, only to find one (or many) of your variables were off.<p>Did the author try multiple instances on each provider? VM tenancy is a bitch. (Think of how annoyed you get at the noise levels, when your neighbor in the next apartment throws a party)<p>Is the author's source benchmarking machine, a physical machine, or a virtualized guest? Does it have power savings turned off so that the process is running at 100% speed, instead of a variably-clocked down core?<p>Did the author enable or disable TCP time-wait recycle? So he doesn't bump into said ceiling when running such tests back to back?<p>Did the author run the tests back to back, or have a cool down period between tests?<p>Where was the author's network upstream located when he tried said tests? Were there any network issues at the time of the test? Would the author even be aware of them?<p>Your page you're testing against, does the same database call, which is presumably cached. Did he throw out the cached results? Can he identify the cached results?<p>Are we firing up ApacheBench with HTTP keepalives enabled? With parallel connections? How many parallel connections?<p>How many Apache servers (StartServers, MinSpareServers, etc)? Which httpd modules were enabled? Which httpd modules were disabled? Which PHP modules were enabled?<p>You're trying to benchmark CPU and I/O horsepower across three different platforms but doing it through this narrow "straw" which consists of your "independent server", your upstream, your upstream's upstream, your upstream's upstream's peering connection with Amazon/Linode/DigitalOcean, your web server and its PHP module, your application, and MySQL.<p>If you're rolling your eyes at this, then you shouldn't be doing weekend benchmarks.<p>I'll leave you with this as well:<p><a href="http://zedshaw.com/essays/programmer_stats.html" rel="nofollow">http://zedshaw.com/essays/programmer_stats.html</a>