Machine Learning, NLP, Computer Vision, Information Retrieval, Transformers.
variables constant. Judging from the results, performance is still not near convergence at the maximum number of parameters, so we can expect to find more interesting phenomena when going even bigger. This is important in real-world settings because smaller models can be distilled or pruned from