Solved by ptrblck in post #2 
              
              
                You could permute the dimensions and use the temporal dimension as a “fake” spatial dimension. 
Afterwards you could permute it back to the original shape. 
Let me know, if this would work for you.