Oh… no it’s not. I’m sorry, I missed your point about Batchnorm here.
I don’t thinks it is possible to use the default batchnorm here. You would need to have your input data projected into same size tensors (some interpolation method?).