Answers.Body: <p>My answer is based on these 2 papers. <a href="https://arxiv.org/pdf/2110.13985" rel="nofollow noreferrer">https://arxiv.org/pdf/2110.13985</a> and <a href="https://arxiv.org/pdf/2111.00396" rel="nofollow noreferrer">https://arxiv.org/pdf/2111.00396</a> and here is the tutorial video <a href="https://www.youtube.com/watch?v=EvQ3ncuriCM" rel="nofollow noreferrer">https://www.youtube.com/watch?v=EvQ3ncuriCM</a>  that explains these two paeprs.</p>
  <p>As they have shown, you can  re-write the sequential operations of RNN in terms of convolutions. Therefore mathematically speaking CNNs can have an equiavalent sequential version that takes one pixel at a time and updates the hidden state and then get the next pixel and so on, until all the pixels of the image is fed to the RNN model.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-09T13:15:57.280Z
Answers.LastActivityDate: 2024-07-10T06:55:06.797Z
Answers.LastEditDate: 2024-07-10T06:55:06.797Z
Answers.UserDisplayName: saleh
Body: <p>I'm trying to distinguish between the fundamental structures of the convolutional neural network and the recurrent neural network. Convolutional neural networks build a <strong>hierarchical</strong> model from the data using the convolutional layers. Recurrent neural networks build a <strong>sequential</strong> model from the data using sequential passing of information (are &quot;recurrent layers&quot; (to contrast with the aforementioned convolutional layers) even a thing? This website seems to have a tag for <code>recurrent-layers</code>, so I guess so?). The differences here aren't clear to me; specifically, it isn't clear to me how one is &quot;hierarchical&quot; whereas the other is &quot;sequential&quot;, precisely how the &quot;hierarchical&quot; structure differs from the &quot;sequential&quot; structures, and how this &quot;hierarchical&quot; vs &quot;sequential&quot; distinction / inductive bias affects the suitability (effectiveness) of the models for different tasks (images vs language is the obvious case, but I'm also interested in this in a more general/abstract sense).</p>
  <p>P.S. With regards to the &quot;recurrent layers&quot; question, I'm familiar with RNN architecture and memory cells (LSTM, GRU), but I've never seen the &quot;recurrent layer&quot; terminology, and it isn't clear to me how any part of the RNN architecture is a &quot;recurrent layer&quot;, as opposed to the convolutional layers of CNNs, which is clearer to me as having distinct &quot;layers&quot; for distinct features (for instance, in an image).</p>
  <p>P.P.S. Given the &quot;sequential&quot; structure, are recurrent neural networks not a hierarchical model? Do they not model things hierarchically? Or do they also model things hierarchically (although, it seems to me that this clearly isn't as clear as with convolutional neural networks)?</p>
  
Tags: <neural-networks><convolutional-neural-networks><recurrent-neural-networks><convolutional-layers><recurrent-layers>
Title: Distinguishing between the fundamental structures of the convolutional neural network and the recurrent neural network: hierarchical vs sequential
UserDisplayName: The Pointer
ThreadText: # Distinguishing between the fundamental structures of the convolutional neural network and the recurrent neural network: hierarchical vs sequential
  
  (Asked by: username_0 on 2024-07-08)
  
  I'm trying to distinguish between the fundamental structures of the convolutional neural network and the recurrent neural network. Convolutional neural networks build a **hierarchical** model from the data using the convolutional layers. Recurrent neural networks build a **sequential** model from the data using sequential passing of information (are "recurrent layers" (to contrast with the aforementioned convolutional layers) even a thing? This website seems to have a tag for `recurrent-layers`, so I guess so?). The differences here aren't clear to me; specifically, it isn't clear to me how one is "hierarchical" whereas the other is "sequential", precisely how the "hierarchical" structure differs from the "sequential" structures, and how this "hierarchical" vs "sequential" distinction / inductive bias affects the suitability (effectiveness) of the models for different tasks (images vs language is the obvious case, but I'm also interested in this in a more general/abstract sense).
  
  P.S. With regards to the "recurrent layers" question, I'm familiar with RNN architecture and memory cells (LSTM, GRU), but I've never seen the "recurrent layer" terminology, and it isn't clear to me how any part of the RNN architecture is a "recurrent layer", as opposed to the convolutional layers of CNNs, which is clearer to me as having distinct "layers" for distinct features (for instance, in an image).
  
  P.P.S. Given the "sequential" structure, are recurrent neural networks not a hierarchical model? Do they not model things hierarchically? Or do they also model things hierarchically (although, it seems to me that this clearly isn't as clear as with convolutional neural networks)?
  
  **Tags:** neural-networks, convolutional-neural-networks, recurrent-neural-networks, convolutional-layers, recurrent-layers
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-07-09. Score: 1 | Accepted answer
  
  My answer is based on these 2 papers. <https://arxiv.org/pdf/2110.13985> and <https://arxiv.org/pdf/2111.00396> and here is the tutorial video <https://www.youtube.com/watch?v=EvQ3ncuriCM> that explains these two paeprs.
  
  As they have shown, you can re-write the sequential operations of RNN in terms of convolutions. Therefore mathematically speaking CNNs can have an equiavalent sequential version that takes one pixel at a time and updates the hidden state and then get the next pixel and so on, until all the pixels of the image is fed to the RNN model.
  
  
  ---
  
  