content.raw: Introducing Zephyr 141B-A35B 🪁: 
content.type: text
content.value: Introducing Zephyr 141B-A35B 🪁: 
content.type: new_line
content.type: new_line
content.raw: https://huggingface.co/HuggingFaceH4/zephyr-orpo-141b-A35b-v0.1
content.id: HuggingFaceH4/zephyr-orpo-141b-A35b-v0.1
content.type: model
content.type: resource
content.url: https://huggingface.co/HuggingFaceH4/zephyr-orpo-141b-A35b-v0.1
content.type: new_line
content.type: new_line
content.raw: Yesterday, Mistral released their latest base model (via magnet link of course 😅) and the community quickly converted it to transformers format and pushed it to the Hub: 
content.type: text
content.value: Yesterday, Mistral released their latest base model (via magnet link of course 😅) and the community quickly converted it to transformers format and pushed it to the Hub: 
content.raw: https://huggingface.co/mistral-community/Mixtral-8x22B-v0.1
content.id: mistral-community/Mixtral-8x22B-v0.1
content.type: model
content.type: resource
content.url: https://huggingface.co/mistral-community/Mixtral-8x22B-v0.1
content.type: new_line
content.type: new_line
content.raw: Early evals of this model looked extremely strong, so we teamed up with Argilla and KAIST AI to cook up a Zephyr recipe with a few new alignment techniques that came out recently:
content.type: text
content.value: Early evals of this model looked extremely strong, so we teamed up with Argilla and KAIST AI to cook up a Zephyr recipe with a few new alignment techniques that came out recently:
content.type: new_line
content.type: new_line
content.raw: 🧑‍🍳 Align the base model with Odds Ratio Preference Optimisation (ORPO). This novel algorithm developed by 
content.type: text
content.value: 🧑‍🍳 Align the base model with Odds Ratio Preference Optimisation (ORPO). This novel algorithm developed by 
content.raw: @JW17
content.type: mention
content.user: JW17
content.raw:  and 
content.type: text
content.value:  and 
content.raw: @nlee-208
content.type: mention
content.user: nlee-208
content.raw:  and 
content.type: text
content.value:  and 
content.raw: @j6mes
content.type: mention
content.user: j6mes
content.raw:  and does not require an SFT step to achieve high performance and is thus much more computationally efficient than methods like DPO and PPO. 
content.type: text
content.value:  and does not require an SFT step to achieve high performance and is thus much more computationally efficient than methods like DPO and PPO. 
content.type: new_line
content.type: new_line
content.raw: 🦫 Use a brand new dataset of 7k high-quality, multi-turn preferences that has been developed by our friends at Argilla. To create this dataset, they took the excellent Capybara SFT dataset from 
content.type: text
content.value: 🦫 Use a brand new dataset of 7k high-quality, multi-turn preferences that has been developed by our friends at Argilla. To create this dataset, they took the excellent Capybara SFT dataset from 
content.raw: @LDJnr
content.type: mention
content.user: LDJnr
content.type: text
content.raw: https://huggingface.co/datasets/LDJnr/Capybara
content.id: LDJnr/Capybara
content.type: dataset
content.type: resource
content.url: https://huggingface.co/datasets/LDJnr/Capybara
content.raw:  and converted it into a preference dataset by augmenting the final turn with responses from new LLMs that were then ranked by GPT-4.
content.type: text
content.value:  and converted it into a preference dataset by augmenting the final turn with responses from new LLMs that were then ranked by GPT-4.
content.type: new_line
content.type: new_line
content.raw: What we find especially neat about this approach is that training on 7k samples only takes ~1.3h on 4 H100 nodes, yet produces a model that is very strong on chat benchmarks like IFEval and BBH.
content.type: text
content.value: What we find especially neat about this approach is that training on 7k samples only takes ~1.3h on 4 H100 nodes, yet produces a model that is very strong on chat benchmarks like IFEval and BBH.
content.type: new_line
content.type: new_line
content.raw: Kudos to 
content.type: text
content.value: Kudos to 
content.raw: @alvarobartt
content.type: mention
content.user: alvarobartt
content.type: text
content.raw: @JW17
content.type: mention
content.user: JW17
content.raw:  and 
content.type: text
content.value:  and 
content.raw: @nlee-208
content.type: mention
content.user: nlee-208
content.raw:  for this very nice and fast-paced collab!
content.type: text
content.value:  for this very nice and fast-paced collab!
content.type: new_line
content.type: new_line
content.raw: For more details on the paper and dataset, checkout our collection: 
content.type: text
content.value: For more details on the paper and dataset, checkout our collection: 
content.raw: https://huggingface.co/collections/HuggingFaceH4/zephyr-orpo-6617eba2c5c0e2cc3c151524
content.id: HuggingFaceH4/zephyr-orpo-6617eba2c5c0e2cc3c151524
content.type: collection
content.type: resource
content.url: https://huggingface.co/collections/HuggingFaceH4/zephyr-orpo-6617eba2c5c0e2cc3c151524
rawContent: Introducing Zephyr 141B-A35B 🪁: 
  
  https://huggingface.co/HuggingFaceH4/zephyr-orpo-141b-A35b-v0.1
  
  Yesterday, Mistral released their latest base model (via magnet link of course 😅) and the community quickly converted it to transformers format and pushed it to the Hub: https://huggingface.co/mistral-community/Mixtral-8x22B-v0.1
  
  Early evals of this model looked extremely strong, so we teamed up with Argilla and KAIST AI to cook up a Zephyr recipe with a few new alignment techniques that came out recently:
  
  🧑‍🍳 Align the base model with Odds Ratio Preference Optimisation (ORPO). This novel algorithm developed by @JW17 and @nlee-208 and @j6mes and does not require an SFT step to achieve high performance and is thus much more computationally efficient than methods like DPO and PPO. 
  
  🦫 Use a brand new dataset of 7k high-quality, multi-turn preferences that has been developed by our friends at Argilla. To create this dataset, they took the excellent Capybara SFT dataset from @LDJnr  https://huggingface.co/datasets/LDJnr/Capybara and converted it into a preference dataset by augmenting the final turn with responses from new LLMs that were then ranked by GPT-4.
  
  What we find especially neat about this approach is that training on 7k samples only takes ~1.3h on 4 H100 nodes, yet produces a model that is very strong on chat benchmarks like IFEval and BBH.
  
  Kudos to @alvarobartt @JW17 and @nlee-208 for this very nice and fast-paced collab!
  
  For more details on the paper and dataset, checkout our collection: https://huggingface.co/collections/HuggingFaceH4/zephyr-orpo-6617eba2c5c0e2cc3c151524
reactions.reaction: ❤️
reactions.users: alvarobartt mehungrymunchie nlee-208 SilentWraith JW17 samusenps hakunamatata1997 dakshdeephere NouRed victor osanseviero avinash02 fynnkroeger dillfrescott clem philschmid srisree rafaelsandroni not-lain netynet
reactions.reaction: 🤗
reactions.users: beta3 clem SilentWraith JW17 mathiasn1 victor osanseviero NickyNicky alvarobartt dillfrescott philschmid not-lain