So the cool thing here is that you can use Spark and TF to find the best model like Microsoft Research did with Resnets.<p><a href="http:&#x2F;&#x2F;www.wired.com&#x2F;2016&#x2F;01&#x2F;microsoft-neural-net-shows-deep-learning-can-get-way-deeper&#x2F;" rel="nofollow">http:&#x2F;&#x2F;www.wired.com&#x2F;2016&#x2F;01&#x2F;microsoft-neural-net-shows-deep...</a><p>They&#x27;re showing you how to train different architectures simultaneously, and then compare their results in order to select the best one. That&#x27;s great as far as it goes.<p>The drawback is that with this schema, you can&#x27;t actually train a given network faster, which is what you want to do with Spark. What is the role of a distributed run-time in training artificial neural networks? It&#x27;s easy. NNs are computationally intensive, so you want to share the work over many machines.<p>Spark can help you orchestrate that through data parallelism, parameter averaging and iterative reduce, which we do with Deeplearning4j.<p><a href="http:&#x2F;&#x2F;deeplearning4j.org&#x2F;spark" rel="nofollow">http:&#x2F;&#x2F;deeplearning4j.org&#x2F;spark</a>
<a href="https:&#x2F;&#x2F;github.com&#x2F;deeplearning4j&#x2F;dl4j-spark-cdh5-examples" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;deeplearning4j&#x2F;dl4j-spark-cdh5-examples</a><p>Data parallelism is an approach Google uses to train neural networks on tons of data quickly. The idea is that you shard your data to a lot of equivalent models, have each of the models train on a separate machine, and then average their parameters. That works, it&#x27;s fast, and it&#x27;s how Spark can help you do deep learning better.