Body: <p>I have a question about the context of CNN and LSTM. I have trained a CNN network for image classification. However, I would like to combine it with LSTM for visualizing the attention weights. So, I extracted the features from the CNN to put it into LSTM. However, I am stuck at the concept of combinating the CNN with LSTM.</p>
  
  <p>– Do I need to train the whole network again? Or just training the LSTM part is fine?
  – Can I just train the LSTM on image sequences based on classes (for e.g. 1 class has around 300 images) and do predictions later on extracted video frames?
  - In what way can I implement the attention mechanism with Keras? </p>
  
  <p>I hope you can help me while I struggle with the context of understanding the combination of this.</p>
  
  <p>~ EDITED ~</p>
  
  <p>I have trained a resnet50 to classify images. Although, I removed the last dense layer, to extract features from the trained CNN network. Those extracted features will be used as input in the newly created LSTM with attention mechanism to find out where the focus lies. The predictions will be on videos (extracted frames).</p>
  
  <p>Image -> extract features (CNN) -> LSTM + Attention (to check where the focus lies during the prediction) -> classify image (output class from N labels) </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-22T17:00:49.840Z
Comments.Text: what is the time (or sequential) component of your system that you want an lstm for? (is it a video? like array of images?)
Comments.UserDisplayName: mshlis
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-22T20:34:54.807Z
Comments.Text: @mshlis Yes it's an array of images
Comments.UserDisplayName: Joker
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-22T23:08:25.230Z
Comments.Text: can you explain more about the CNN you trained and what you want to transfer over for what goal?
Comments.UserDisplayName: mshlis
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-22T23:29:51.783Z
Comments.Text: @mshlis updated the question. See ~ EDITED part ~.
Comments.UserDisplayName: Joker
Tags: <neural-networks><convolutional-neural-networks><python><keras><attention>
Title: Understanding CNN+LSTM concept with attention and need help
UserDisplayName: Joker
ThreadText: # Understanding CNN+LSTM concept with attention and need help
  
  (Asked by: username_0 on 2019-07-22)
  
  I have a question about the context of CNN and LSTM. I have trained a CNN network for image classification. However, I would like to combine it with LSTM for visualizing the attention weights. So, I extracted the features from the CNN to put it into LSTM. However, I am stuck at the concept of combinating the CNN with LSTM.
  
  – Do I need to train the whole network again? Or just training the LSTM part is fine?
  – Can I just train the LSTM on image sequences based on classes (for e.g. 1 class has around 300 images) and do predictions later on extracted video frames?
  - In what way can I implement the attention mechanism with Keras?
  
  I hope you can help me while I struggle with the context of understanding the combination of this.
  
  ~ EDITED ~
  
  I have trained a resnet50 to classify images. Although, I removed the last dense layer, to extract features from the trained CNN network. Those extracted features will be used as input in the newly created LSTM with attention mechanism to find out where the focus lies. The predictions will be on videos (extracted frames).
  
  Image -> extract features (CNN) -> LSTM + Attention (to check where the focus lies during the prediction) -> classify image (output class from N labels)
  
  **Tags:** neural-networks, convolutional-neural-networks, python, keras, attention
  
  > **Comments:**
  > **username_1**: what is the time (or sequential) component of your system that you want an lstm for? (is it a video? like array of images?)  
  > **username_0**: @username_1 Yes it's an array of images  
  > **username_1**: can you explain more about the CNN you trained and what you want to transfer over for what goal?  
  > **username_0**: @username_1 updated the question. See ~ EDITED part ~.  
  
  ---
  
  ## 0 answers
  
  