How to add task to the leaderboard?

#921
by alvations - opened

Hi all, we're interested to add some multilingual/translation tasks to the leaderboard, esp. with datasets covered in https://github.com/mjpost/sacrebleu

Does anyone have pointers on how do we go about cloning the leaderboard and editing the app files to incorporate new tasks?

Thank you in advance for the pointers!

Regards,
Liling

Open LLM Leaderboard org

Hi @alvations ,

Thank you for your interest! @clefourrier has more expertise in this matter and I hope she will be able to help you and provide instructions next week after her holiday. We appreciate your patience!

If you have any other questions, feel free to open a new discussion, I hope I can help! ☺️

Open LLM Leaderboard org

Hi!

We've got a documentation page on how to create your own leaderboard here !
Do you already know which evaluation suite you'll want to use?

Feel free to send me an email to discuss and I can add you to our slack to give you a hand

clefourrier changed discussion status to closed

Sign up or log in to comment