<p>T0 starts with T5+LM as the base model but then performs multi-task finetuning on it using BIG-Bench tasks. As detailed in the Results section (Section 6) of their <a href="https://arxiv.org/pdf/2110.08207.pdf" rel="nofollow noreferrer">paper</a>, T0 outperforms T5+LM on a number of tasks.</p>

CC BY-SA 4.0
2024-04-28T00:46:57.470Z
But didn't T5 also train on multiple tasks?
HappyFace
CC BY-SA 4.0
2023-04-12T06:13:44.903Z
2023-04-12T06:13:44.903Z
Harsh
<p>What is the difference between T5 and T0 models? I had read that T0 is T5 + LM. But as I know T5 uses encoder-decoder model like BART but BART can be used as LM so that's mean that T5 has a LM function too. So what is the difference between T5 and T0 then?</p>

CC BY-SA 4.0
2023-04-13T20:01:15.297Z
Can you provide more context? Where did you find these terms? Provide a link.
nbro
<deep-learning><natural-language-processing><gpt><language-model>
What is the difference between T5 and T0 models?
prostak
# What is the difference between T5 and T0 models?

(Asked by: username_0 on 2023-04-11)

What is the difference between T5 and T0 models? I had read that T0 is T5 + LM. But as I know T5 uses encoder-decoder model like BART but BART can be used as LM so that's mean that T5 has a LM function too. So what is the difference between T5 and T0 then?

**Tags:** deep-learning, natural-language-processing, gpt, language-model

> **Comments:**
> **username_1**: Can you provide more context? Where did you find these terms? Provide a link.  

---

## 1 answers

### Answer by username_2 on 2023-04-12. Score: 2 | Accepted answer

T0 starts with T5+LM as the base model but then performs multi-task finetuning on it using BIG-Bench tasks. As detailed in the Results section (Section 6) of their [paper](https://arxiv.org/pdf/2110.08207.pdf), T0 outperforms T5+LM on a number of tasks.

> **Comments:**
> **username_3**: But didn't T5 also train on multiple tasks?  

---

