@ptrblck : At the same time, we normally call backward on the loss instead of the output correct?. Does it mean that in a real network for example, we will see the backward pass of batchnorm not applying the channels last format?