content.raw: Can we improve the quality of open LLMs for more languages?
content.type: text
content.value: Can we improve the quality of open LLMs for more languages?
content.type: new_line
content.type: new_line
content.raw: Step 1: Evaluate current SOTA.
content.type: text
content.value: Step 1: Evaluate current SOTA.
content.type: new_line
content.type: new_line
content.raw: The Data Is Better Together community has rated more than 10K prompts for quality. We now want to translate a subset of these to help address the language gap in model evals.
content.type: text
content.value: The Data Is Better Together community has rated more than 10K prompts for quality. We now want to translate a subset of these to help address the language gap in model evals.
content.type: new_line
content.type: new_line
content.raw: The plan is roughly this:
content.type: text
content.value: The plan is roughly this:
content.type: new_line
content.type: new_line
content.raw: - We started with 
content.type: text
content.value: - We started with 
content.href: https://huggingface.co/datasets/DIBT/10k_prompts_ranked
content.raw: https://huggingface.co/datasets/DIBT/10k_prompts_ranked
content.type: link
content.raw:  and took a subset of 500 high-quality prompts
content.type: text
content.value:  and took a subset of 500 high-quality prompts
content.type: new_line
content.raw: - We're asking the community to translate these prompts into different languages
content.type: text
content.value: - We're asking the community to translate these prompts into different languages
content.type: new_line
content.raw: - We'll evaluate the extent to which we can use AlpacaEval and similar approaches to rate the outputs of models across these different languages
content.type: text
content.value: - We'll evaluate the extent to which we can use AlpacaEval and similar approaches to rate the outputs of models across these different languages
content.type: new_line
content.raw: - If it works well, we can more easily evaluate open LLMs across different languages by using a judge LLM to rate the quality of outputs from different models.
content.type: text
content.value: - If it works well, we can more easily evaluate open LLMs across different languages by using a judge LLM to rate the quality of outputs from different models.
content.type: new_line
content.type: new_line
content.raw: You can find more details in our new GitHub repo: 
content.type: text
content.value: You can find more details in our new GitHub repo: 
content.href: https://github.com/huggingface/data-is-better-together
content.raw: https://github.com/huggingface/data-is-better-together
content.type: link
content.raw:  (don't forget to give it a ⭐!) 
content.type: text
content.value:  (don't forget to give it a ⭐!) 
rawContent: Can we improve the quality of open LLMs for more languages?
  
  Step 1: Evaluate current SOTA.
  
  The Data Is Better Together community has rated more than 10K prompts for quality. We now want to translate a subset of these to help address the language gap in model evals.
  
  The plan is roughly this:
  
  - We started with https://huggingface.co/datasets/DIBT/10k_prompts_ranked and took a subset of 500 high-quality prompts
  - We're asking the community to translate these prompts into different languages
  - We'll evaluate the extent to which we can use AlpacaEval and similar approaches to rate the outputs of models across these different languages
  - If it works well, we can more easily evaluate open LLMs across different languages by using a judge LLM to rate the quality of outputs from different models.
  
  You can find more details in our new GitHub repo: https://github.com/huggingface/data-is-better-together (don't forget to give it a ⭐!) 
reactions.reaction: ❤️
reactions.users: dvilasuero clefourrier samusenps sarahooker Joseph717171
reactions.reaction: 🤗
reactions.users: dvilasuero sarahooker Joseph717171 seyf1elislam
reactions.reaction: 🚀
reactions.users: dvilasuero sarahooker Joseph717171
reactions.reaction: 🔥
reactions.users: dvilasuero sarahooker Joseph717171
reactions.reaction: 🤝
reactions.users: victor sarahooker Joseph717171