The primary consideration then is how much previous information is important in your model’s current decision? Say we are at time t, does the state of the video at time t-1, t-2, … matter in your solution?
If not, then this is a simple image classification problem. You can imagine every frame where there is no click to be one in which the outcome is 0 and each from where there is a click to be one where the outcome is 1. When your model predicts 1, you can also predict the x and y coordinates of your outcome.
Now lets say this is your case. What you can do is load up any number of frames and use a standard ResNet to classify each frame as needing a click or not. Unless there is a need for a different type of model, this is where I’d start.
Once you have a model that is converging for predicting which frames need to be clicked, I would then take all frames that need click and use a resnet, this time with a pair of linear outcomes, to predict what the x and y corrdinates of the click need to be.
Now if you need information from previous frames, you’re going to need to look into some sequence modeling type approaches like an RNN. This is a little trickier, but overall not much. There seem to be a good number of examples on the internet that can help with that problem.