Solved by Igor_Susmelj in post #2 
              
              
                If I understand you correctly, you try to train a LSTM network on extracted features using VGG. And you only want to train the LSTM model, not the VGG one. 
Why don’t you extract the features of all the frames using VGG and store them in a file. And then train the LSTM model on the extracted feature…