content.raw: We've just published a detailed blog post on the creation of Cosmopedia dataset. We hope this will provide insights about generating synthetic data at scale for pre-training. 
content.type: text
content.value: We've just published a detailed blog post on the creation of Cosmopedia dataset. We hope this will provide insights about generating synthetic data at scale for pre-training. 
content.type: new_line
content.href: https://huggingface.co/blog/cosmopedia
content.raw: https://huggingface.co/blog/cosmopedia
content.type: link
content.type: new_line
content.type: new_line
content.raw: Here are some key takeaways:  
content.type: text
content.value: Here are some key takeaways:  
content.type: new_line
content.raw: 🎯 Prompt curation is crucial: we want to cover many topics with few duplicates.
content.type: text
content.value: 🎯 Prompt curation is crucial: we want to cover many topics with few duplicates.
content.type: new_line
content.raw: 📚 You can leverage various resources for diversity: using different seed data, generation formats, and target audiences.
content.type: text
content.value: 📚 You can leverage various resources for diversity: using different seed data, generation formats, and target audiences.
content.type: new_line
content.raw: ⚙️ The importance of a good technical stack: for scalable generations with tools like llm-swarm and fast model training and evaluation.
content.type: text
content.value: ⚙️ The importance of a good technical stack: for scalable generations with tools like llm-swarm and fast model training and evaluation.
content.type: new_line
content.type: new_line
content.raw: Have a good read!
content.type: text
content.value: Have a good read!
rawContent: We've just published a detailed blog post on the creation of Cosmopedia dataset. We hope this will provide insights about generating synthetic data at scale for pre-training. 
  https://huggingface.co/blog/cosmopedia
  
  Here are some key takeaways:  
  🎯 Prompt curation is crucial: we want to cover many topics with few duplicates.
  📚 You can leverage various resources for diversity: using different seed data, generation formats, and target audiences.
  ⚙️ The importance of a good technical stack: for scalable generations with tools like llm-swarm and fast model training and evaluation.
  
  Have a good read!
reactions.reaction: 🤗
reactions.users: alielfilali01 anton-l vwxyzjn kavalciemre samusenps osanseviero lvwerra xu3kev avinash02 thomwolf fpaupier manu muhtasham mjoshi malmenea AdinaY victor clefourrier molonelaveh
reactions.reaction: ❤️
reactions.users: alielfilali01 anton-l vwxyzjn samusenps osanseviero xu3kev mmhamdy thomwolf muhtasham aloobun Salwa-Zeitoun AdinaY Dflare molonelaveh timpearce svallory
reactions.reaction: 🔥
reactions.users: alielfilali01 anton-l vwxyzjn severo osanseviero xu3kev mmhamdy thomwolf muhtasham AdinaY ajibawa-2023 InferenceIllusionist mlabonne on1onmangoes molonelaveh
reactions.reaction: 🤯
reactions.users: svallory
commentators._id: 5fcaabed246881afd5b00167
commentators.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1651847561574-5fcaabed246881afd5b00167.jpeg
commentators.fullname: Muhtasham Oblokulov
commentators.name: muhtasham
commentators.type: user