BLOOM (language model)
BigScience Large Open-science Open-access Multilingual Language Model (BLOOM) is a transformer-based large language model. It was created by over 1,000 AI researchers to provide a free large language model for large-scale public access. Trained on around 366 billion tokens over March through July 2022, it is considered an alternative to OpenAI's GPT-3 with its 176 billion parameters. BLOOM uses a decoder-only transformer model architecture modified from Megatron-LM GPT-2. The BLOOM project was started by a co-founder of Hugging Face. Six main groups of people were involved, including HuggingFace's BigScience team, the Microsoft DeepSpeed team, the NVIDIA Megatron-LM team,  the IDRIS/GENCI team, the PyTorch team, and the volunteers in the BigScience Engineering workgroup. BLOOM was trained using data of 46 natural languages and 13 programming languages. In total, 1.6 terabytes of pre-processed text was converted into 350 billion unique tokens as BLOOM's training datasets. References 

Large language models
Generative pre-trained transformers