content.raw: [New crazy blog post alert] We are releasing an extensive blog post on the science of creating high quality web-scale datasets, detailing all the steps and learnings that came in our recent 15 trillion tokens 🍷FineWeb release
content.type: text
content.value: [New crazy blog post alert] We are releasing an extensive blog post on the science of creating high quality web-scale datasets, detailing all the steps and learnings that came in our recent 15 trillion tokens 🍷FineWeb release
content.type: new_line
content.type: new_line
content.raw: Inspired by the distill.pub interactive graphics papers, we settled to write the most extensive, enjoyable and in-depth tech report we could draft on so prepare for a 45-mmin read with interactive graphics and all.
content.type: text
content.value: Inspired by the distill.pub interactive graphics papers, we settled to write the most extensive, enjoyable and in-depth tech report we could draft on so prepare for a 45-mmin read with interactive graphics and all.
content.type: new_line
content.type: new_line
content.raw: And it's not all, in this article we also introduce 📚FineWeb-Edu a filtered subset of Common Crawl with 1.3T tokens containing only web pages with very high educational content. Up to our knowledge, FineWeb-Edu out-performs all openly release web-scale datasets by a significant margin on knowledge- and reasoning-intensive benchmarks like MMLU, ARC, and OpenBookQA
content.type: text
content.value: And it's not all, in this article we also introduce 📚FineWeb-Edu a filtered subset of Common Crawl with 1.3T tokens containing only web pages with very high educational content. Up to our knowledge, FineWeb-Edu out-performs all openly release web-scale datasets by a significant margin on knowledge- and reasoning-intensive benchmarks like MMLU, ARC, and OpenBookQA
content.type: new_line
content.type: new_line
content.raw: We also make a number of surprising observations on the "quality" of the internet it-self which may challenge some of the general assumptions on web data (not saying more, I'll let you draw your conclusions ;)
content.type: text
content.value: We also make a number of surprising observations on the "quality" of the internet it-self which may challenge some of the general assumptions on web data (not saying more, I'll let you draw your conclusions ;)
content.type: new_line
content.type: new_line
content.raw: https://huggingface.co/spaces/HuggingFaceFW/blogpost-fineweb-v1
content.id: HuggingFaceFW/blogpost-fineweb-v1
content.type: space
content.type: resource
content.url: https://huggingface.co/spaces/HuggingFaceFW/blogpost-fineweb-v1
rawContent: [New crazy blog post alert] We are releasing an extensive blog post on the science of creating high quality web-scale datasets, detailing all the steps and learnings that came in our recent 15 trillion tokens 🍷FineWeb release
  
  Inspired by the distill.pub interactive graphics papers, we settled to write the most extensive, enjoyable and in-depth tech report we could draft on so prepare for a 45-mmin read with interactive graphics and all.
  
  And it's not all, in this article we also introduce 📚FineWeb-Edu a filtered subset of Common Crawl with 1.3T tokens containing only web pages with very high educational content. Up to our knowledge, FineWeb-Edu out-performs all openly release web-scale datasets by a significant margin on knowledge- and reasoning-intensive benchmarks like MMLU, ARC, and OpenBookQA
  
  We also make a number of surprising observations on the "quality" of the internet it-self which may challenge some of the general assumptions on web data (not saying more, I'll let you draw your conclusions ;)
  
  https://huggingface.co/spaces/HuggingFaceFW/blogpost-fineweb-v1
reactions.reaction: 🔥
reactions.users: loubnabnl mmhamdy leonardlin KingNish KonradSzafer SixOpen Joseph717171 dingo-actual fffiloni louisbrulenaudet ngxson SaylorTwift clem adamelliotfields titan087 adhisetiawan ahmadele dadaddy p20p
reactions.reaction: 🚀
reactions.users: loubnabnl mmhamdy mmoy KonradSzafer rreed-pha Joseph717171 SaylorTwift clem
reactions.reaction: ❤️
reactions.users: afrideva Joseph717171 medmekk clem dadaddy
reactions.reaction: 🤗
reactions.users: mmhamdy Joseph717171 clem OmbelineM
commentators._id: 5e67bdd61009063689407479
commentators.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1583857146757-5e67bdd61009063689407479.jpeg
commentators.fullname: Clem 🤗
commentators.name: clem
commentators.type: user