Hi,
About the ordering, I think NCHW is much more intuitive rather than latter choice. It is like going from high level to low level view (batch_size > patch_size > channel_size).
PyTorch uses a Storage for each tensor that follows a particular layout 12. As PyTorch uses strided layout for mapping logical view to the physical location of data in the memory, there should not be any difference in performance as it is only a swap of channels in stride value.  There are some information about this in a talk by @ezyang in his blog post PyTorch internals : Inside 245-5D 26).
In a part of it, this image has been provided and a caption that says: By the way, we’re interested in making this picture not true; instead of having a separate concept of storage, just define a view to be a tensor that is backed by a base tensor. This is a little more complicated, but it has the benefit that contiguous tensors get a much more direct representation without the Storage indirection. A change like this would make PyTorch’s internal representation a bit more like Numpy’s.
image1600×1200 84.5 KB
But in the link you have provided, it has been shown that there is a perf gain in almost all prominent architectures. What could be the possible explanation for this?
Bests