Can we improve the quality of open LLMs for more languages?
text
Can we improve the quality of open LLMs for more languages?
new_line
new_line
Step 1: Evaluate current SOTA.
text
Step 1: Evaluate current SOTA.
new_line
new_line
The Data Is Better Together community has rated more than 10K prompts for quality. We now want to translate a subset of these to help address the language gap in model evals.
text
The Data Is Better Together community has rated more than 10K prompts for quality. We now want to translate a subset of these to help address the language gap in model evals.
new_line
new_line
The plan is roughly this:
text
The plan is roughly this:
new_line
new_line
- We started with 
text
- We started with 
https://huggingface.co/datasets/DIBT/10k_prompts_ranked
https://huggingface.co/datasets/DIBT/10k_prompts_ranked
link
 and took a subset of 500 high-quality prompts
text
 and took a subset of 500 high-quality prompts
new_line
- We're asking the community to translate these prompts into different languages
text
- We're asking the community to translate these prompts into different languages
new_line
- We'll evaluate the extent to which we can use AlpacaEval and similar approaches to rate the outputs of models across these different languages
text
- We'll evaluate the extent to which we can use AlpacaEval and similar approaches to rate the outputs of models across these different languages
new_line
- If it works well, we can more easily evaluate open LLMs across different languages by using a judge LLM to rate the quality of outputs from different models.
text
- If it works well, we can more easily evaluate open LLMs across different languages by using a judge LLM to rate the quality of outputs from different models.
new_line
new_line
You can find more details in our new GitHub repo: 
text
You can find more details in our new GitHub repo: 
https://github.com/huggingface/data-is-better-together
https://github.com/huggingface/data-is-better-together
link
 (don't forget to give it a ⭐!) 
text
 (don't forget to give it a ⭐!) 
Can we improve the quality of open LLMs for more languages?

Step 1: Evaluate current SOTA.

The Data Is Better Together community has rated more than 10K prompts for quality. We now want to translate a subset of these to help address the language gap in model evals.

The plan is roughly this:

- We started with https://huggingface.co/datasets/DIBT/10k_prompts_ranked and took a subset of 500 high-quality prompts
- We're asking the community to translate these prompts into different languages
- We'll evaluate the extent to which we can use AlpacaEval and similar approaches to rate the outputs of models across these different languages
- If it works well, we can more easily evaluate open LLMs across different languages by using a judge LLM to rate the quality of outputs from different models.

You can find more details in our new GitHub repo: https://github.com/huggingface/data-is-better-together (don't forget to give it a ⭐!) 
❤️
dvilasuero clefourrier samusenps sarahooker Joseph717171
🤗
dvilasuero sarahooker Joseph717171 seyf1elislam
🚀
dvilasuero sarahooker Joseph717171
🔥
dvilasuero sarahooker Joseph717171
🤝
victor sarahooker Joseph717171