content.raw: 🚨 We’ve just released a new tool to compare the performance of models in the 🤗 Open LLM Leaderboard: the Comparator 🎉
content.type: text
content.value: 🚨 We’ve just released a new tool to compare the performance of models in the 🤗 Open LLM Leaderboard: the Comparator 🎉
content.type: new_line
content.raw: https://huggingface.co/spaces/open-llm-leaderboard/comparator
content.id: open-llm-leaderboard/comparator
content.type: space
content.type: resource
content.url: https://huggingface.co/spaces/open-llm-leaderboard/comparator
content.type: new_line
content.type: new_line
content.raw: Want to see how two different versions of LLaMA stack up? Let’s walk through a step-by-step comparison of LLaMA-3.1 and LLaMA-3.2. 🦙🧵👇
content.type: text
content.value: Want to see how two different versions of LLaMA stack up? Let’s walk through a step-by-step comparison of LLaMA-3.1 and LLaMA-3.2. 🦙🧵👇
content.type: new_line
content.type: new_line
content.raw: 1/ Load the Models' Results
content.type: text
content.value: 1/ Load the Models' Results
content.type: new_line
content.raw: - Go to the 🤗 Open LLM Leaderboard Comparator: 
content.type: text
content.value: - Go to the 🤗 Open LLM Leaderboard Comparator: 
content.raw: https://huggingface.co/spaces/open-llm-leaderboard/comparator
content.id: open-llm-leaderboard/comparator
content.type: space
content.type: resource
content.url: https://huggingface.co/spaces/open-llm-leaderboard/comparator
content.type: new_line
content.raw: - Search for "LLaMA-3.1" and "LLaMA-3.2" in the model dropdowns.
content.type: text
content.value: - Search for "LLaMA-3.1" and "LLaMA-3.2" in the model dropdowns.
content.type: new_line
content.raw: - Press the Load button. Ready to dive into the results!
content.type: text
content.value: - Press the Load button. Ready to dive into the results!
content.type: new_line
content.type: new_line
content.raw: 2/ Compare Metric Results in the Results Tab 📊
content.type: text
content.value: 2/ Compare Metric Results in the Results Tab 📊
content.type: new_line
content.raw: - Head over to the Results tab.
content.type: text
content.value: - Head over to the Results tab.
content.type: new_line
content.raw: - Here, you’ll see the performance metrics for each model, beautifully color-coded using a gradient to highlight performance differences: greener is better! 🌟
content.type: text
content.value: - Here, you’ll see the performance metrics for each model, beautifully color-coded using a gradient to highlight performance differences: greener is better! 🌟
content.type: new_line
content.raw: - Want to focus on a specific task? Use the Task filter to hone in on comparisons for tasks like BBH or MMLU-Pro.
content.type: text
content.value: - Want to focus on a specific task? Use the Task filter to hone in on comparisons for tasks like BBH or MMLU-Pro.
content.type: new_line
content.type: new_line
content.raw: 3/ Check Config Alignment in the Configs Tab ⚙️
content.type: text
content.value: 3/ Check Config Alignment in the Configs Tab ⚙️
content.type: new_line
content.raw: - To ensure you’re comparing apples to apples, head to the Configs tab.
content.type: text
content.value: - To ensure you’re comparing apples to apples, head to the Configs tab.
content.type: new_line
content.raw: - Review both models’ evaluation configurations, such as metrics, datasets, prompts, few-shot configs...
content.type: text
content.value: - Review both models’ evaluation configurations, such as metrics, datasets, prompts, few-shot configs...
content.type: new_line
content.raw: - If something looks off, it’s good to know before drawing conclusions! ✅
content.type: text
content.value: - If something looks off, it’s good to know before drawing conclusions! ✅
content.type: new_line
content.type: new_line
content.raw: 4/ Compare Predictions by Sample in the Details Tab 🔍
content.type: text
content.value: 4/ Compare Predictions by Sample in the Details Tab 🔍
content.type: new_line
content.raw: - Curious about how each model responds to specific inputs? The Details tab is your go-to!
content.type: text
content.value: - Curious about how each model responds to specific inputs? The Details tab is your go-to!
content.type: new_line
content.raw: - Select a Task (e.g., MuSR) and then a Subtask (e.g., Murder Mystery) and then press the Load Details button.
content.type: text
content.value: - Select a Task (e.g., MuSR) and then a Subtask (e.g., Murder Mystery) and then press the Load Details button.
content.type: new_line
content.raw: - Check out the side-by-side predictions and dive into the nuances of each model’s outputs.
content.type: text
content.value: - Check out the side-by-side predictions and dive into the nuances of each model’s outputs.
content.type: new_line
content.type: new_line
content.raw: 5/ With this tool, it’s never been easier to explore how small changes between model versions affect performance on a wide range of tasks. Whether you’re a researcher or enthusiast, you can instantly visualize improvements and dive into detailed comparisons.
content.type: text
content.value: 5/ With this tool, it’s never been easier to explore how small changes between model versions affect performance on a wide range of tasks. Whether you’re a researcher or enthusiast, you can instantly visualize improvements and dive into detailed comparisons.
content.type: new_line
content.type: new_line
content.raw: 🚀 Try the 🤗 Open LLM Leaderboard Comparator now and take your model evaluations to the next level!
content.type: text
content.value: 🚀 Try the 🤗 Open LLM Leaderboard Comparator now and take your model evaluations to the next level!
rawContent: 🚨 We’ve just released a new tool to compare the performance of models in the 🤗 Open LLM Leaderboard: the Comparator 🎉
  https://huggingface.co/spaces/open-llm-leaderboard/comparator
  
  Want to see how two different versions of LLaMA stack up? Let’s walk through a step-by-step comparison of LLaMA-3.1 and LLaMA-3.2. 🦙🧵👇
  
  1/ Load the Models' Results
  - Go to the 🤗 Open LLM Leaderboard Comparator: https://huggingface.co/spaces/open-llm-leaderboard/comparator
  - Search for "LLaMA-3.1" and "LLaMA-3.2" in the model dropdowns.
  - Press the Load button. Ready to dive into the results!
  
  2/ Compare Metric Results in the Results Tab 📊
  - Head over to the Results tab.
  - Here, you’ll see the performance metrics for each model, beautifully color-coded using a gradient to highlight performance differences: greener is better! 🌟
  - Want to focus on a specific task? Use the Task filter to hone in on comparisons for tasks like BBH or MMLU-Pro.
  
  3/ Check Config Alignment in the Configs Tab ⚙️
  - To ensure you’re comparing apples to apples, head to the Configs tab.
  - Review both models’ evaluation configurations, such as metrics, datasets, prompts, few-shot configs...
  - If something looks off, it’s good to know before drawing conclusions! ✅
  
  4/ Compare Predictions by Sample in the Details Tab 🔍
  - Curious about how each model responds to specific inputs? The Details tab is your go-to!
  - Select a Task (e.g., MuSR) and then a Subtask (e.g., Murder Mystery) and then press the Load Details button.
  - Check out the side-by-side predictions and dive into the nuances of each model’s outputs.
  
  5/ With this tool, it’s never been easier to explore how small changes between model versions affect performance on a wide range of tasks. Whether you’re a researcher or enthusiast, you can instantly visualize improvements and dive into detailed comparisons.
  
  🚀 Try the 🤗 Open LLM Leaderboard Comparator now and take your model evaluations to the next level!
reactions.reaction: 👍
reactions.users: m4r14n3s akjindal53244 DmitryRyumin dvilasuero gabrielmbmb Joseph717171 louisbrulenaudet
reactions.reaction: 👀
reactions.users: John6666 Joseph717171