Deep Learning, Artificial Intelligence, Llm, NLP, Language Model.
training FLOPs and requires no new data sources, making it highly efficient and convenient. The UL2 objective combines prefix language modeling and long-short span corruption (e.g., infilling) tasks, that can teach LLM to leverage bidirectional attention (i.e., PrefixLM) or leverage infilling-style