Besides being the coolest named benchmark in the game, HellaSwag is an important measurement of здравый смысль (or common sense) in LLMs.
text
Besides being the coolest named benchmark in the game, HellaSwag is an important measurement of здравый смысль (or common sense) in LLMs.
new_line
new_line
- More on HellaSwag: 
text
- More on HellaSwag: 
https://github.com/rowanz/hellaswag
https://github.com/rowanz/hellaswag
link
new_line
new_line
I spent the afternoon benchmarking YandexGPT Pro 4th Gen, one of the Russian tech giant's premier models.
text
I spent the afternoon benchmarking YandexGPT Pro 4th Gen, one of the Russian tech giant's premier models.
new_line
new_line
- Yandex HF Org: 
text
- Yandex HF Org: 
https://cdn-avatars.huggingface.co/v1/production/uploads/1655918403280-609baae0fe087f3d04cf0481.jpeg
https://huggingface.co/yandex
yandex
org
resource
https://huggingface.co/yandex
new_line
- More on Yandex models: 
text
- More on Yandex models: 
https://yandex.cloud/ru/docs/foundation-models/concepts/yandexgpt/models
https://yandex.cloud/ru/docs/foundation-models/concepts/yandexgpt/models
link
new_line
new_line
The eval notebook is available on GitHub and the resulting dataset is already on the HF Hub!
text
The eval notebook is available on GitHub and the resulting dataset is already on the HF Hub!
new_line
new_line
- Eval Notebook: 
text
- Eval Notebook: 
https://github.com/kghamilton89/ai-explorer/blob/main/yandex-hellaswag/hellaswag-assess.ipynb
https://github.com/kghamilton89/ai-explorer/blob/main/yandex-hellaswag/hellaswag-assess.ipynb
link
new_line
- Eval Dataset: 
text
- Eval Dataset: 
https://huggingface.co/datasets/ZennyKenny/yandexgptpro_4th_gen-hellaswag
ZennyKenny/yandexgptpro_4th_gen-hellaswag
dataset
resource
https://huggingface.co/datasets/ZennyKenny/yandexgptpro_4th_gen-hellaswag
new_line
new_line
And of course, everyone wants to see the results so have a look at the results in the context of other zero-shot experiments that I was able to find!
text
And of course, everyone wants to see the results so have a look at the results in the context of other zero-shot experiments that I was able to find!
Besides being the coolest named benchmark in the game, HellaSwag is an important measurement of здравый смысль (or common sense) in LLMs.

- More on HellaSwag: https://github.com/rowanz/hellaswag

I spent the afternoon benchmarking YandexGPT Pro 4th Gen, one of the Russian tech giant's premier models.

- Yandex HF Org: https://huggingface.co/yandex
- More on Yandex models: https://yandex.cloud/ru/docs/foundation-models/concepts/yandexgpt/models

The eval notebook is available on GitHub and the resulting dataset is already on the HF Hub!

- Eval Notebook: https://github.com/kghamilton89/ai-explorer/blob/main/yandex-hellaswag/hellaswag-assess.ipynb
- Eval Dataset: https://huggingface.co/datasets/ZennyKenny/yandexgptpro_4th_gen-hellaswag

And of course, everyone wants to see the results so have a look at the results in the context of other zero-shot experiments that I was able to find!
❤️
PujiPramusanto modenn Nggajr xanzzzz kuma21 xinnn63 John6666 limaku willdinata risxyie Takugen dantezxcd on3dj
🔥
loqhunter Abru dantezxcd
➕
31d332y0v3rj0y3d dantezxcd
6758a9850e3fff481964ca6d
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/EolfJfjW25hC4Bt_hCPq8.png
Jean Louis
JLouisBiz
user
656e3808d4de03a07d116850
https://cdn-avatars.huggingface.co/v1/production/uploads/656e3808d4de03a07d116850/62cFw46AmuhdI3gS24F1M.jpeg
Kenneth Hamilton
ZennyKenny
user