Hi Bram,
Thanks for your reply. That’s not exactly what I am trying to solve, sorry for not posting the original problem. Actually, I am trying to do Scene segmentation for Movies, in which shots’ representations are given, I have to predict if a shot is the ending of a scene; where the scene boundary detection could be regarded as a binary classification problem on shot boundaries.
Please check this link:

  
      docs.google.com
  
  
    

Eluvio DS/ML Challenge 2021 4

Scene Segmentation Using the MovieScenes Dataset Introduction Scenes are critical units in movies, containing a composition of people, places, objects, and actions/activities. Different from individual tasks in a single domain, scene segmentation...

  
  
    
    
  
  


I want to use a RNN or a BERT model for which I want to input the representations of the shots i.e. like embeddings for each word. I want to make use of the output of each RNN hidden unit (with each shot as input).