Answers.Body: <p>Yes, it is possible to use deep learning architecture with Apache Spark now. Databricks have Spark-deep-learning which is a pipeline based in python and uses tensorflow and keras.</p>
  
  <p><a href="https://github.com/databricks/spark-deep-learning" rel="nofollow noreferrer">https://github.com/databricks/spark-deep-learning</a></p>
  
  <p>You can check this. There is also BigDL by Intel analytics.
  <a href="https://github.com/intel-analytics/BigDL" rel="nofollow noreferrer">https://github.com/intel-analytics/BigDL</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-26T16:34:19.987Z
Answers.LastActivityDate: 2020-02-26T16:34:19.987Z
Answers.UserDisplayName: overflow95
Body: <p>If it is possible, will it be really useful or the model will end up converging very early(with a typical optimum learning rate) ? Any content on this topic will be helpful for me.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-29T03:50:54.697Z
Comments.Text: Apache Spark is open source, general-purpose distributed computing engine used for processing and analyzing a large amount of data. Just like Hadoop MapReduce, it also works with the system to distribute data across the cluster and process the data in parallel.
Comments.UserDisplayName: Sharath
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-26T09:08:29.490Z
Comments.Text: For those not familiar with it, could you explain what "spark" is?
Comments.UserDisplayName: Oliver Mason
Tags: <neural-networks><deep-learning><tensorflow><convergence>
Title: Is it possible to use deeplearning with spark (with a distributed databases as HDFS or Cassandra)?
UserDisplayName: Sharath
ThreadText: # Is it possible to use deeplearning with spark (with a distributed databases as HDFS or Cassandra)?
  
  (Asked by: username_0 on 2020-02-26)
  
  If it is possible, will it be really useful or the model will end up converging very early(with a typical optimum learning rate) ? Any content on this topic will be helpful for me.
  
  **Tags:** neural-networks, deep-learning, tensorflow, convergence
  > **Comments:**
  > **username_1**: For those not familiar with it, could you explain what "spark" is?  
  > **username_0**: Apache Spark is open source, general-purpose distributed computing engine used for processing and analyzing a large amount of data. Just like Hadoop MapReduce, it also works with the system to distribute data across the cluster and process the data in parallel.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-02-26. Score: 2 | Accepted answer
  
  Yes, it is possible to use deep learning architecture with Apache Spark now. Databricks have Spark-deep-learning which is a pipeline based in python and uses tensorflow and keras.
  
  <https://github.com/databricks/spark-deep-learning>
  
  You can check this. There is also BigDL by Intel analytics.
  <https://github.com/intel-analytics/BigDL>
  
  
  ---
  
  