<p>Yes, it is possible to use deep learning architecture with Apache Spark now. Databricks have Spark-deep-learning which is a pipeline based in python and uses tensorflow and keras.</p>

<p><a href="https://github.com/databricks/spark-deep-learning" rel="nofollow noreferrer">https://github.com/databricks/spark-deep-learning</a></p>

<p>You can check this. There is also BigDL by Intel analytics.
<a href="https://github.com/intel-analytics/BigDL" rel="nofollow noreferrer">https://github.com/intel-analytics/BigDL</a></p>

CC BY-SA 4.0
2020-02-26T16:34:19.987Z
2020-02-26T16:34:19.987Z
overflow95
<p>If it is possible, will it be really useful or the model will end up converging very early(with a typical optimum learning rate) ? Any content on this topic will be helpful for me.</p>

CC BY-SA 4.0
2020-02-29T03:50:54.697Z
Apache Spark is open source, general-purpose distributed computing engine used for processing and analyzing a large amount of data. Just like Hadoop MapReduce, it also works with the system to distribute data across the cluster and process the data in parallel.
Sharath
CC BY-SA 4.0
2020-02-26T09:08:29.490Z
For those not familiar with it, could you explain what "spark" is?
Oliver Mason
<neural-networks><deep-learning><tensorflow><convergence>
Is it possible to use deeplearning with spark (with a distributed databases as HDFS or Cassandra)?
Sharath
# Is it possible to use deeplearning with spark (with a distributed databases as HDFS or Cassandra)?

(Asked by: username_0 on 2020-02-26)

If it is possible, will it be really useful or the model will end up converging very early(with a typical optimum learning rate) ? Any content on this topic will be helpful for me.

**Tags:** neural-networks, deep-learning, tensorflow, convergence

> **Comments:**
> **username_1**: For those not familiar with it, could you explain what "spark" is?  
> **username_0**: Apache Spark is open source, general-purpose distributed computing engine used for processing and analyzing a large amount of data. Just like Hadoop MapReduce, it also works with the system to distribute data across the cluster and process the data in parallel.  

---

## 1 answers

### Answer by username_2 on 2020-02-26. Score: 2 | Accepted answer

Yes, it is possible to use deep learning architecture with Apache Spark now. Databricks have Spark-deep-learning which is a pipeline based in python and uses tensorflow and keras.

<https://github.com/databricks/spark-deep-learning>

You can check this. There is also BigDL by Intel analytics.
<https://github.com/intel-analytics/BigDL>


---

