Deep Learning, Artificial Intelligence, Image Classification, Vision Transformer, Transformers.
Space and SuperNet Search space consists of: (i) the depth of the network, measured by the number of blocks Nj per stage, (ii) the width of the network, i.e., the channel dimension Cj per stage, and (iii) expansion ratio Ei,j of each FFN. Elastic depth can be naturally implemented through