Thank you @ptrblck
Now it’s working. Yes, that’s a possible solution I tried and it worked. But instead disturbing the i3d architecture I converted the output of network into[batch_size, 1] by max-pooling with the dimension of 1 and then squeezed the output which makes my target and output shape the same.