You’re right that many backends will use NCHW, but the conversions usually have no runtime cost – the graph optimizer completely eliminates transposes that cancel.  I just brought up an experimental NCHW backend and no transposes were needed to run ResNet or ResNeXt :-).