So just to clarify, your input is a video sequence and your output is a classification of where and when there is a click on the video? Do you have multiple videos, or just one video? Are you trying to model the video as a sequence, or individual frames?