Machine Learning, NLP, Computer Vision, Information Retrieval, Transformers.
the authors call Structural Re-parametrization. This method allows for training a model with residual connections and then converting them into a single-path model topology that makes inference wicked fast. The ImageNet Top-1 performance is still in the 80%s (far from SOTA’s 90%), but the real