content.raw: I've run the open llm leaderboard evaluations + hellaswag on 
content.type: text
content.value: I've run the open llm leaderboard evaluations + hellaswag on 
content.raw: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B
content.id: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
content.type: model
content.type: resource
content.url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B
content.raw:  and compared to 
content.type: text
content.value:  and compared to 
content.raw: https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct
content.id: meta-llama/Llama-3.1-8B-Instruct
content.type: model
content.type: resource
content.url: https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct
content.raw:  and at first glance R1 do not beat Llama overall.
content.type: text
content.value:  and at first glance R1 do not beat Llama overall.
content.type: new_line
content.type: new_line
content.raw: If anyone wants to double check the results are posted here: 
content.type: text
content.value: If anyone wants to double check the results are posted here: 
content.type: new_line
content.href: https://github.com/csabakecskemeti/lm_eval_results
content.raw: https://github.com/csabakecskemeti/lm_eval_results
content.type: link
content.type: new_line
content.type: new_line
content.raw: Am I made some mistake, or (at least this distilled version) not as good/better than the competition?
content.type: text
content.value: Am I made some mistake, or (at least this distilled version) not as good/better than the competition?
content.type: new_line
content.type: new_line
content.raw: I'll run the same on the Qwen 7B distilled version too.
content.type: text
content.value: I'll run the same on the Qwen 7B distilled version too.
content.type: new_line
content.type: text
rawContent: I've run the open llm leaderboard evaluations + hellaswag on https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B and compared to https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct and at first glance R1 do not beat Llama overall.
  
  If anyone wants to double check the results are posted here: 
  https://github.com/csabakecskemeti/lm_eval_results
  
  Am I made some mistake, or (at least this distilled version) not as good/better than the competition?
  
  I'll run the same on the Qwen 7B distilled version too.
   
reactions.reaction: 👀
reactions.users: John6666 djuna KingNish PilotSB fredbarre fakezeta
commentators._id: 659be8bbb0f43ed69f17e7b8
commentators.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/659be8bbb0f43ed69f17e7b8/FXMz0Qabx16c5Jr-vusPT.jpeg
commentators.fullname: SB
commentators.name: shb777
commentators.type: user
commentators._id: 64f0462031fba2b85ccfac22
commentators.avatarUrl: /avatars/4184829a85c4ee7c74071ffc257c01d5.svg
commentators.fullname: huangbin
commentators.name: bin110
commentators.type: user
commentators._id: 64e6d37e02dee9bcb9d9fa18
commentators.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64e6d37e02dee9bcb9d9fa18/os24VYiNCoyth9yQSdv_A.jpeg
commentators.fullname: Csaba  Kecskemeti
commentators.name: csabakecskemeti
commentators.type: user