Yash_Jakhotiya:

Why does PyTorch favor channels first?


For historic performance reasons, if I’m not mistaken.



 Yash_Jakhotiya:

If it’s a matter of convention, shouldn’t there be an easy way to choose your convention?
If it’s because CuDNN prefers NCHW, can PyTorch not internally change input?


PyTorch can change the memory layout via to(memory_format=torch.channels_last), which is the preferred format for mixed-precision convolutions using TensorCores.



 Yash_Jakhotiya:

Why would you want devs to explicitly convert their input and their models? Can we not have a parameter in model declaration that decides channels_first or channels_last and the input is processed accordingly?


Wouldn’t this already be the case or are you concerned about having to add the to() operation to the input as well? Under the hood, the parameter layout might be respected and the inputs thus transformed for you, but explicit transformations are preferred over implicit ones.