NLP, Artificial Intelligence, Data Science, Machine Learning, Neural Networks.
Two minutes NLP — Leveraging parallelisms to train large neural networks Pipeline, tensor, and data parallelism to train the MT-NLG model Hello fellow NLP enthusiasts! The race towards ever larger language models continues and this article will focus on the type of parallelisms used to make it