Answers.Body: <p>T0 starts with T5+LM as the base model but then performs multi-task finetuning on it using BIG-Bench tasks. As detailed in the Results section (Section 6) of their <a href="https://arxiv.org/pdf/2110.08207.pdf" rel="nofollow noreferrer">paper</a>, T0 outperforms T5+LM on a number of tasks.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-28T00:46:57.470Z
Answers.Text: But didn't T5 also train on multiple tasks?
Answers.UserDisplayName: HappyFace
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-12T06:13:44.903Z
Answers.LastActivityDate: 2023-04-12T06:13:44.903Z
Answers.UserDisplayName: Harsh
Body: <p>What is the difference between T5 and T0 models? I had read that T0 is T5 + LM. But as I know T5 uses encoder-decoder model like BART but BART can be used as LM so that's mean that T5 has a LM function too. So what is the difference between T5 and T0 then?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-04-13T20:01:15.297Z
Comments.Text: Can you provide more context? Where did you find these terms? Provide a link.
Comments.UserDisplayName: nbro
Tags: <deep-learning><natural-language-processing><gpt><language-model>
Title: What is the difference between T5 and T0 models?
UserDisplayName: prostak
ThreadText: # What is the difference between T5 and T0 models?
  
  (Asked by: username_0 on 2023-04-11)
  
  What is the difference between T5 and T0 models? I had read that T0 is T5 + LM. But as I know T5 uses encoder-decoder model like BART but BART can be used as LM so that's mean that T5 has a LM function too. So what is the difference between T5 and T0 then?
  
  **Tags:** deep-learning, natural-language-processing, gpt, language-model
  > **Comments:**
  > **username_1**: Can you provide more context? Where did you find these terms? Provide a link.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-04-12. Score: 2 | Accepted answer
  
  T0 starts with T5+LM as the base model but then performs multi-task finetuning on it using BIG-Bench tasks. As detailed in the Results section (Section 6) of their [paper](https://arxiv.org/pdf/2110.08207.pdf), T0 outperforms T5+LM on a number of tasks.
  
  > **Comments:**
  > **username_3**: But didn't T5 also train on multiple tasks?  
  
  ---
  
  