Sorry for the late reply.
Actually the frames are extracted from a random video and there is no label for the object classification of each frame. I just tried to use the deep features to see if I can find the similarity between frames and find the boundary of videos. Just like you said, the model is not trained on my dataset so I think maybe the pre-trained resnet is not suitable in my case. Thank you so much for helping me to figure out the reason.