a more generic function, which in itself can also be a network. In contrast to “Network-in-Network” [26] that turns
out to increase the dimension of depth, we show that our
“Network-in- Neuron ” expands along a new dimension.
Formally, we present aggregated transformations as:
F(x) =CX
i=1Ti(x); (2)
whereTi(x)can be an arbitrary function. Analogous to a
simple neuron,Tishould project xinto an (optionally lowdimensional) embedding and then transform it.
In Eqn.(2), Cis the size of the set of transformations
to be aggregated. We refer to Cascardinality [2]. In
Eqn.(2) Cis in a position similar to Din Eqn.(1), but C
need not equal Dand can be an arbitrary number. While
the dimension of width is related to the number of simple
transformations (inner product), we argue that the dimension of cardinality controls the number of more complex
transformations. We show by experiments that cardinality
is an essential dimension and can be more effective than the
dimensions of width and depth.
In this paper, we consider a simple way of designing the
transformation functions: all Ti’s have the same topology.
We present a simple, highly modularized network architecture for image
classification. Our network is constructed by repeating a building block that
aggregates a set of transformations with the same topology. Our simple design
results in a homogeneous, multi-branch architecture that has only a few
hyper-parameters to set. This strategy exposes a new dimension, which we call
"cardinality" (the size of the set of transformations), as an essential factor
in addition to the dimensions of depth and width. On the ImageNet-1K dataset,
we empirically show that even under the restricted condition of maintaining
complexity, increasing cardinality is able to improve classification accuracy.
Moreover, increasing cardinality is more effective than going deeper or wider
when we increase the capacity. Our models, named ResNeXt, are the foundations
of our entry to the ILSVRC 2016 classification task in which we secured 2nd
place. We further investigate ResNeXt on an ImageNet-5K set and the COCO
detection set, also showing better results than its ResNet counterpart. The
code and models are publicly available online.
Saining Xie Ross Girshick Piotr Dollár Zhuowen Tu Kaiming He