Machine Learning, NLP, Computer Vision, Information Retrieval, Transformers.
applications. The main theme behind this work is “let’s go back to the basics and excel at that”: they brag about not having branching, only using 3x3 convolutions + ReLUs and not using architecture search, compound scaling nor any other “heavy designs”. The main contribution of this paper is what