As someone who has done research in AI, you can train all the state of the art models with a single computer (couple of TitanX GPUs, top of the line CPU, couple terabyte SSD, 32 GB RAM) that any engineer can afford.<p>Contrary to popular belief, state of the art deep learning is not commonly run on multi-node clusters. Although hardware itself is not the bottleneck for innovation in the current state of the art in deep learning, if we restrict ourself to  hardware, the bottleneck is memory bandwidth.