the dimensions of depth and width.
Residual connections. The following table shows the effects of the residual (shortcut) connections:
setting w/ residual w/o residual
ResNet-50 164d 23.9 31.2
ResNeXt-50 324d 22.2 26.1
Removing shortcuts from the ResNeXt-50 increases the error by 3.9 points to 26.1%. Removing shortcuts from its
ResNet-50 counterpart is much worse (31.2%). These comparisons suggest that the residual connections are helpful
foroptimization , whereas aggregated transformations are
stronger representations , as shown by the fact that they
perform consistently better than their counterparts with or
without residual connections.
Performance. For simplicity we use Torch’s built-in
grouped convolution implementation, without special optimization. We note that this implementation was brute-force
and not parallelization-friendly. On 8 GPUs of NVIDIA
M40, training 324d ResNeXt-101 in Table 3 takes 0.95s
per mini-batch, vs. 0.70s of ResNet-101 baseline that has
similar FLOPs. We argue that this is a reasonable overhead.
We expect carefully engineered lower-level implementation
(e.g., in CUDA) will reduce this overhead. We also expect
We present a simple, highly modularized network architecture for image
classification. Our network is constructed by repeating a building block that
aggregates a set of transformations with the same topology. Our simple design
results in a homogeneous, multi-branch architecture that has only a few
hyper-parameters to set. This strategy exposes a new dimension, which we call
"cardinality" (the size of the set of transformations), as an essential factor
in addition to the dimensions of depth and width. On the ImageNet-1K dataset,
we empirically show that even under the restricted condition of maintaining
complexity, increasing cardinality is able to improve classification accuracy.
Moreover, increasing cardinality is more effective than going deeper or wider
when we increase the capacity. Our models, named ResNeXt, are the foundations
of our entry to the ILSVRC 2016 classification task in which we secured 2nd
place. We further investigate ResNeXt on an ImageNet-5K set and the COCO
detection set, also showing better results than its ResNet counterpart. The
code and models are publicly available online.
Saining Xie Ross Girshick Piotr Dollár Zhuowen Tu Kaiming He