There is no such thing as a &#x27;large commodity cluster.&#x27;  You can order a pile of commodity parts and spend the next five years fiddling with it as you watch your metrics fall into the toilet, or you can contract with a systems integrator with domain experience to make sure that you can actually accomplish the work you set out to do.  When you&#x27;re moving an established set of software from a known environment into a new cluster, it helps to have a wide array of technical experts and SLAs in place to ensure that the environment -- here meaning the physical compute infrastructure, storage service, and software installation -- is reliable and maintainable, so that your team can focus on their core mission -- in this case, the modeling software.<p>&quot;Chaos monkey&quot; works fine when you&#x27;re dealing with random cloudstuff and you are in a position to duck-punch your production code.  When you have a large dataset that you need to process reliably and predictably, the mantra shifts to &quot;fail never&quot;, and that requires engineering up front in combination with carefully-planned maintenance.<p>Full disclosure:  I&#x27;ve worked on this specific supercomputer.  While I&#x27;m not on the admin team in question, I do work with them occasionally and I have a lot of respect for them -- their job is not an easy one.