Answers.Body: <p>There are tasks in computer vision where recurrent neural networks (RNNs) can be useful because there's some sequential sub-task in the main task.</p>
  <p>For instance, in the paper <a href="https://openaccess.thecvf.com/content_cvpr_2015/html/Donahue_Long-Term_Recurrent_Convolutional_2015_CVPR_paper.html" rel="nofollow noreferrer">Long-Term Recurrent Convolutional Networks for Visual Recognition and Description</a>, the authors investigate the use of a neural network that is both recurrent and convolutional to solve certain computer vision tasks that also have a sequential component/part, such as <strong>video recognition tasks</strong>, <strong>image to sentence generation problems</strong>, and <strong>video narration challenges</strong>.</p>
  <p>There are other papers that investigate the combination of convolutional and recurrent layers, such as <a href="https://www.frontiersin.org/articles/10.3389/fpsyg.2017.01551/full" rel="nofollow noreferrer">Recurrent Convolutional Neural Networks: A Better Model of Biological Object Recognition</a>, which also has a biological motivation.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-07T12:57:29.543Z
Answers.LastActivityDate: 2020-07-07T12:57:29.543Z
Answers.UserDisplayName: nbro
Body: <p>I am learning computer vision. When I was going through implementations of various computer vision projects, some OCR problems used GRU or LSTM, while some did not. I understand that RNNs are used only in problems where input data is a sequence, like audio or text.</p>
  <p>So, in <a href="https://www.kaggle.com/c/digit-recognizer/notebooks" rel="nofollow noreferrer">kernels of MNIST on kaggle</a> almost no kernel has used RNNs and almost every repository for <a href="https://github.com/search?q=iam%20handwriting" rel="nofollow noreferrer">OCR on IAM dataset on GitHub</a> has used GRU or LSTMs. Intuitively, written text in an image is a sequence, so RNNs were used. But, so is the written text in MNIST data. So, when exactly is it that RNNs(or GRUs or LSTMs) need to be used in computer vision and when don't?</p>
  
Tags: <deep-learning><convolutional-neural-networks><computer-vision><recurrent-neural-networks><r-cnn>
Title: Why are RNNs used in some computer vision problems?
UserDisplayName: Naveen Reddy Marthala
ThreadText: # Why are RNNs used in some computer vision problems?
  
  (Asked by: username_0 on 2020-07-06)
  
  I am learning computer vision. When I was going through implementations of various computer vision projects, some OCR problems used GRU or LSTM, while some did not. I understand that RNNs are used only in problems where input data is a sequence, like audio or text.
  
  So, in [kernels of MNIST on kaggle](https://www.kaggle.com/c/digit-recognizer/notebooks) almost no kernel has used RNNs and almost every repository for [OCR on IAM dataset on GitHub](https://github.com/search?q=iam%20handwriting) has used GRU or LSTMs. Intuitively, written text in an image is a sequence, so RNNs were used. But, so is the written text in MNIST data. So, when exactly is it that RNNs(or GRUs or LSTMs) need to be used in computer vision and when don't?
  
  **Tags:** deep-learning, convolutional-neural-networks, computer-vision, recurrent-neural-networks, r-cnn
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-07-07. Score: 3
  
  There are tasks in computer vision where recurrent neural networks (RNNs) can be useful because there's some sequential sub-task in the main task.
  
  For instance, in the paper [Long-Term Recurrent Convolutional Networks for Visual Recognition and Description](https://openaccess.thecvf.com/content_cvpr_2015/html/Donahue_Long-Term_Recurrent_Convolutional_2015_CVPR_paper.html), the authors investigate the use of a neural network that is both recurrent and convolutional to solve certain computer vision tasks that also have a sequential component/part, such as **video recognition tasks**, **image to sentence generation problems**, and **video narration challenges**.
  
  There are other papers that investigate the combination of convolutional and recurrent layers, such as [Recurrent Convolutional Neural Networks: A Better Model of Biological Object Recognition](https://www.frontiersin.org/articles/10.3389/fpsyg.2017.01551/full), which also has a biological motivation.
  
  
  ---
  
  