content.raw: ⚓️ Sailor: A New Multilingual Open LLM for South-East Asia 🌏
content.type: text
content.value: ⚓️ Sailor: A New Multilingual Open LLM for South-East Asia 🌏
content.type: new_line
content.type: new_line
content.raw: Last month we have released a new family of multilingual language models called **Sailor**, ranging from 0.5B to 7B parameters, continually pre-trained from the Qwen1.5 models. Based on our extensive benchmarking, the Sailor models demonstrate exceptional performance on South-East Asian languages, taking us one step closer to multilingual LLMs that can serve the diverse needs of the region and beyond. 
content.type: text
content.value: Last month we have released a new family of multilingual language models called **Sailor**, ranging from 0.5B to 7B parameters, continually pre-trained from the Qwen1.5 models. Based on our extensive benchmarking, the Sailor models demonstrate exceptional performance on South-East Asian languages, taking us one step closer to multilingual LLMs that can serve the diverse needs of the region and beyond. 
content.type: new_line
content.type: new_line
content.raw: Today, we're more than excited to share the key technical details behind the Sailor models! 💪
content.type: text
content.value: Today, we're more than excited to share the key technical details behind the Sailor models! 💪
content.type: new_line
content.type: new_line
content.raw: **Key highlights**:
content.type: text
content.value: **Key highlights**:
content.type: new_line
content.raw: 🔍 Data curation: Merging short examples, document-level code-switching, aggressive data cleaning and deduplication.
content.type: text
content.value: 🔍 Data curation: Merging short examples, document-level code-switching, aggressive data cleaning and deduplication.
content.type: new_line
content.raw: 🤖 Tokenization Robustness: We find that BPE dropout is really effective to deal with prompt variations.
content.type: text
content.value: 🤖 Tokenization Robustness: We find that BPE dropout is really effective to deal with prompt variations.
content.type: new_line
content.raw: 🔍 Optimizing Data Mixture: We propose a new approach to automatically balance capabilities across different languages! 
content.type: text
content.value: 🔍 Optimizing Data Mixture: We propose a new approach to automatically balance capabilities across different languages! 
content.type: new_line
content.raw: 🌟 Recipe in Continual Pre-training: We discover a powerful metric that can help predict how well the Sailor models will perform on the original domain (e.g., English) after continual pre-training.
content.type: text
content.value: 🌟 Recipe in Continual Pre-training: We discover a powerful metric that can help predict how well the Sailor models will perform on the original domain (e.g., English) after continual pre-training.
content.type: new_line
content.type: new_line
content.raw: We are thrilled to share these technical details with the community and invite you to explore the Sailor models. We hope Sailor models take us one step closer to multilingual LLMs in the world! 🌍✨ 
content.type: text
content.value: We are thrilled to share these technical details with the community and invite you to explore the Sailor models. We hope Sailor models take us one step closer to multilingual LLMs in the world! 🌍✨ 
content.type: new_line
content.type: new_line
content.raw: To learn more, please access our research paper or reach out to our team.
content.type: text
content.value: To learn more, please access our research paper or reach out to our team.
content.type: new_line
content.raw: 🔗 Paper: 
content.type: text
content.value: 🔗 Paper: 
content.label: Sailor: Open Language Models for South-East Asia (2404.03608)
content.raw: https://huggingface.co/papers/2404.03608
content.id: 2404.03608
content.type: paper
content.type: resource
content.url: https://huggingface.co/papers/2404.03608
content.type: new_line
content.raw: 🧩 Model: 
content.type: text
content.value: 🧩 Model: 
content.raw: https://huggingface.co/collections/sail/sailor-language-models-65e19a749f978976f1959825
content.id: sail/sailor-language-models-65e19a749f978976f1959825
content.type: collection
content.type: resource
content.url: https://huggingface.co/collections/sail/sailor-language-models-65e19a749f978976f1959825
content.type: new_line
content.raw: 💻 Code: 
content.type: text
content.value: 💻 Code: 
content.href: https://github.com/sail-sg/sailor-llm
content.raw: https://github.com/sail-sg/sailor-llm
content.type: link
content.type: new_line
rawContent: ⚓️ Sailor: A New Multilingual Open LLM for South-East Asia 🌏
  
  Last month we have released a new family of multilingual language models called **Sailor**, ranging from 0.5B to 7B parameters, continually pre-trained from the Qwen1.5 models. Based on our extensive benchmarking, the Sailor models demonstrate exceptional performance on South-East Asian languages, taking us one step closer to multilingual LLMs that can serve the diverse needs of the region and beyond. 
  
  Today, we're more than excited to share the key technical details behind the Sailor models! 💪
  
  **Key highlights**:
  🔍 Data curation: Merging short examples, document-level code-switching, aggressive data cleaning and deduplication.
  🤖 Tokenization Robustness: We find that BPE dropout is really effective to deal with prompt variations.
  🔍 Optimizing Data Mixture: We propose a new approach to automatically balance capabilities across different languages! 
  🌟 Recipe in Continual Pre-training: We discover a powerful metric that can help predict how well the Sailor models will perform on the original domain (e.g., English) after continual pre-training.
  
  We are thrilled to share these technical details with the community and invite you to explore the Sailor models. We hope Sailor models take us one step closer to multilingual LLMs in the world! 🌍✨ 
  
  To learn more, please access our research paper or reach out to our team.
  🔗 Paper: https://huggingface.co/papers/2404.03608
  🧩 Model: https://huggingface.co/collections/sail/sailor-language-models-65e19a749f978976f1959825
  💻 Code: https://github.com/sail-sg/sailor-llm
  
reactions.reaction: 🔥
reactions.users: dreamerdeo Jiiiia teddythinh chaoscodes DmitryRyumin taufiqdp SivilTaram osanseviero clem victor lunarflu Kernel nikgr gabrielchua DongfuJiang damerajee yjernite
reactions.reaction: 🚀
reactions.users: dreamerdeo Jiiiia teddythinh aboutyourresearch chaoscodes monsoon-nlp osanseviero clem SivilTaram samusenps lunarflu damerajee yjernite
reactions.reaction: ❤️
reactions.users: vitaliim0 damerajee
reactions.reaction: 👀
reactions.users: SimonRAVCARE