content.raw: 𝗟𝗹𝗮𝗺𝗮-𝟯.𝟭 𝗺𝗼𝗱𝗲𝗹𝘀 𝗳𝗶𝗻𝗮𝗹𝗹𝘆 𝗴𝗲𝘁 𝘁𝗵𝗲𝗶𝗿 𝗖𝗵𝗮𝘁𝗯𝗼𝘁 𝗔𝗿𝗲𝗻𝗮 𝗿𝗮𝗻𝗸𝗶𝗻𝗴 🎖️
content.type: text
content.value: 𝗟𝗹𝗮𝗺𝗮-𝟯.𝟭 𝗺𝗼𝗱𝗲𝗹𝘀 𝗳𝗶𝗻𝗮𝗹𝗹𝘆 𝗴𝗲𝘁 𝘁𝗵𝗲𝗶𝗿 𝗖𝗵𝗮𝘁𝗯𝗼𝘁 𝗔𝗿𝗲𝗻𝗮 𝗿𝗮𝗻𝗸𝗶𝗻𝗴 🎖️
content.type: new_line
content.type: new_line
content.raw: Given the impressive benchmarks published my Meta for their Llama-3.1 models, I was curious to see how these models would compare to top proprietary models on Chatbot Arena.
content.type: text
content.value: Given the impressive benchmarks published my Meta for their Llama-3.1 models, I was curious to see how these models would compare to top proprietary models on Chatbot Arena.
content.type: new_line
content.type: new_line
content.raw: Now we've got the results! LMSys released the ELO derived from thousands of user votes for the new models, and here are the rankings:
content.type: text
content.value: Now we've got the results! LMSys released the ELO derived from thousands of user votes for the new models, and here are the rankings:
content.type: new_line
content.type: new_line
content.raw: 💥 405B Model ranks 5th overall, in front of GPT-4-turbo! But behind GPT-4o, Claude-3.5 Sonnet and Gemini-advanced.
content.type: text
content.value: 💥 405B Model ranks 5th overall, in front of GPT-4-turbo! But behind GPT-4o, Claude-3.5 Sonnet and Gemini-advanced.
content.type: new_line
content.raw: 👏 70B Model climbs up to 9th rank ! From 1206 ➡️ 1244.
content.type: text
content.value: 👏 70B Model climbs up to 9th rank ! From 1206 ➡️ 1244.
content.type: new_line
content.raw: 👍 8B Model improves from 1152 ➡️ 1170.
content.type: text
content.value: 👍 8B Model improves from 1152 ➡️ 1170.
content.type: new_line
content.type: new_line
content.raw: ✅ This confirms that Llama-3.1 is a good contender for any task: any of its 3 model size is much cheaper to run than equivalent proprietary models!
content.type: text
content.value: ✅ This confirms that Llama-3.1 is a good contender for any task: any of its 3 model size is much cheaper to run than equivalent proprietary models!
content.type: new_line
content.type: new_line
content.raw: For instance, here are the inference prices for the top models;
content.type: text
content.value: For instance, here are the inference prices for the top models;
content.type: new_line
content.raw: ➤ GPT-4-Turbo inference price from OpenAI: $5/M input tokens, $15/M output tokens
content.type: text
content.value: ➤ GPT-4-Turbo inference price from OpenAI: $5/M input tokens, $15/M output tokens
content.type: new_line
content.raw: ➤ Llama-3.1-405B from HF API (for testing only): 3$/M for input or output tokens (Source linked in the first comment)
content.type: text
content.value: ➤ Llama-3.1-405B from HF API (for testing only): 3$/M for input or output tokens (Source linked in the first comment)
content.type: new_line
content.raw: ➤ Llama-3.1-405B from HF API (for testing only): free ✨
content.type: text
content.value: ➤ Llama-3.1-405B from HF API (for testing only): free ✨
content.type: new_line
content.type: new_line
content.raw: Get a head start on the HF API (resource by 
content.type: text
content.value: Get a head start on the HF API (resource by 
content.raw: @andrewrreed
content.type: mention
content.user: andrewrreed
content.raw: ) 👉 
content.type: text
content.value: ) 👉 
content.href: https://huggingface.co/learn/cookbook/enterprise_hub_serverless_inference_api
content.raw: https://huggingface.co/learn/cookbook/enterprise_hub_serverless_inference_api
content.type: link
rawContent: 𝗟𝗹𝗮𝗺𝗮-𝟯.𝟭 𝗺𝗼𝗱𝗲𝗹𝘀 𝗳𝗶𝗻𝗮𝗹𝗹𝘆 𝗴𝗲𝘁 𝘁𝗵𝗲𝗶𝗿 𝗖𝗵𝗮𝘁𝗯𝗼𝘁 𝗔𝗿𝗲𝗻𝗮 𝗿𝗮𝗻𝗸𝗶𝗻𝗴 🎖️
  
  Given the impressive benchmarks published my Meta for their Llama-3.1 models, I was curious to see how these models would compare to top proprietary models on Chatbot Arena.
  
  Now we've got the results! LMSys released the ELO derived from thousands of user votes for the new models, and here are the rankings:
  
  💥 405B Model ranks 5th overall, in front of GPT-4-turbo! But behind GPT-4o, Claude-3.5 Sonnet and Gemini-advanced.
  👏 70B Model climbs up to 9th rank ! From 1206 ➡️ 1244.
  👍 8B Model improves from 1152 ➡️ 1170.
  
  ✅ This confirms that Llama-3.1 is a good contender for any task: any of its 3 model size is much cheaper to run than equivalent proprietary models!
  
  For instance, here are the inference prices for the top models;
  ➤ GPT-4-Turbo inference price from OpenAI: $5/M input tokens, $15/M output tokens
  ➤ Llama-3.1-405B from HF API (for testing only): 3$/M for input or output tokens (Source linked in the first comment)
  ➤ Llama-3.1-405B from HF API (for testing only): free ✨
  
  Get a head start on the HF API (resource by @andrewrreed) 👉 https://huggingface.co/learn/cookbook/enterprise_hub_serverless_inference_api
reactions.reaction: 🔥
reactions.users: Stoneking89 louisbrulenaudet Tonic andrewrreed
commentators._id: 6569216f9c96f1a47bf45788
commentators.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6569216f9c96f1a47bf45788/mCLqmAs4dOjKdxNQVAp1w.png
commentators.fullname: Sica Rius
commentators.name: SicariusSicariiStuff
commentators.type: user