title: Fully Open Source Moxin-7B Technical Report
summary: Recently, Large Language Models (LLMs) have undergone a significant
  transformation, marked by a rapid rise in both their popularity and
  capabilities. Leading this evolution are proprietary LLMs like GPT-4 and
  GPT-o1, which have captured widespread attention in the AI community due to
  their remarkable performance and versatility. Simultaneously, open-source LLMs,
  such as LLaMA and Mistral, have made great contributions to the ever-increasing
  popularity of LLMs due to the ease to customize and deploy the models across
  diverse applications. Although open-source LLMs present unprecedented
  opportunities for innovation and research, the commercialization of LLMs has
  raised concerns about transparency, reproducibility, and safety. Many
  open-source LLMs fail to meet fundamental transparency requirements by
  withholding essential components like training code and data, and some use
  restrictive licenses whilst claiming to be "open-source," which may hinder
  further innovations on LLMs. To mitigate this issue, we introduce Moxin 7B, a
  fully open-source LLM developed in accordance with the Model Openness Framework
  (MOF), a ranked classification system that evaluates AI models based on model
  completeness and openness, adhering to principles of open science, open source,
  open data, and open access. Our model achieves the highest MOF classification
  level of "open science" through the comprehensive release of pre-training code
  and configurations, training and fine-tuning datasets, and intermediate and
  final checkpoints. Experiments show that our model achieves superior
  performance in zero-shot evaluation compared with popular 7B models and
  performs competitively in few-shot evaluation.
paper_authors._id: 6758ff6ea90b1c382daf3526
paper_authors.name: Pu Zhao
paper_authors._id: 6758ff6ea90b1c382daf3527
paper_authors.name: Xuan Shen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-25T08:22:14.683Z
paper_authors._id: 65f7197c66afd39dd7f58522
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/zx-6RlvxTATmIO3dCIYez.png
paper_authors.fullname: Shawn Shen
paper_authors.type: user
paper_authors.user: shawnricecake
paper_authors._id: 6758ff6ea90b1c382daf3528
paper_authors.name: Zhenglun Kong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-20T10:54:54.577Z
paper_authors._id: 5f2c36551ebc8c6ede2f0e53
paper_authors.avatarUrl: /avatars/e3ddbd15f50b86958377b5fc2460a57e.svg
paper_authors.fullname: Tony Kong
paper_authors.type: user
paper_authors.user: TonyK
paper_authors._id: 6758ff6ea90b1c382daf3529
paper_authors.name: Yixin Shen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-11T13:54:16.438Z
paper_authors._id: 65e02dcd121aa0d0b4a95963
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/6C5bq1weQxI9s80u3Z8v1.jpeg
paper_authors.fullname: Yixin
paper_authors.type: user
paper_authors.user: yixinshen1
paper_authors._id: 6758ff6ea90b1c382daf352a
paper_authors.name: Sung-En Chang
paper_authors._id: 6758ff6ea90b1c382daf352b
paper_authors.name: Timothy Rupprecht
paper_authors._id: 6758ff6ea90b1c382daf352c
paper_authors.name: Lei Lu
paper_authors._id: 6758ff6ea90b1c382daf352d
paper_authors.name: Enfu Nan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-11T13:54:31.486Z
paper_authors._id: 65382338ef7d2bb8c4b373bc
paper_authors.avatarUrl: /avatars/a469924cf3b983c6e420ee4dccce0290.svg
paper_authors.fullname: ENFU NAN
paper_authors.type: user
paper_authors.user: nanenfu
paper_authors._id: 6758ff6ea90b1c382daf352e
paper_authors.name: Changdi Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-11T13:54:43.185Z
paper_authors._id: 654321438caf77017835e5c0
paper_authors.avatarUrl: /avatars/999e63e33e17058baa2ce53d90f49d63.svg
paper_authors.fullname: yangchangdi
paper_authors.type: user
paper_authors.user: yangchangdi
paper_authors._id: 6758ff6ea90b1c382daf352f
paper_authors.name: Yumei He
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-11T13:54:51.471Z
paper_authors._id: 66c9ec0edf893af1ae3329b3
paper_authors.avatarUrl: /avatars/0a09c5ad9e56bee64eb098d893b1c969.svg
paper_authors.fullname: Yumei He
paper_authors.type: user
paper_authors.user: Joy2024HuggingFace
paper_authors._id: 6758ff6ea90b1c382daf3530
paper_authors.name: Xingchen Xu
paper_authors._id: 6758ff6ea90b1c382daf3531
paper_authors.name: Yu Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-11T13:55:07.226Z
paper_authors._id: 6463d44a9dd8b530a85fe6c9
paper_authors.avatarUrl: /avatars/24364533d8e3ccec20c0043c98967f35.svg
paper_authors.fullname: Yu Huang
paper_authors.type: user
paper_authors.user: yuhuang
paper_authors._id: 6758ff6ea90b1c382daf3532
paper_authors.name: Wei Wang
paper_authors._id: 6758ff6ea90b1c382daf3533
paper_authors.name: Yue Chen
paper_authors._id: 6758ff6ea90b1c382daf3534
paper_authors.name: Yong He
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-11T13:53:21.591Z
paper_authors._id: 674f84d2b2c5d446d8fcbe72
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Vpq1zbM_8hZX9YENKnC-Q.png
paper_authors.fullname: Yong He
paper_authors.type: user
paper_authors.user: heyong4725
paper_authors._id: 6758ff6ea90b1c382daf3535
paper_authors.name: Yanzhi Wang
paper_title: Fully Open Source Moxin-7B Technical Report
paper_summary: Recently, Large Language Models (LLMs) have undergone a significant
  transformation, marked by a rapid rise in both their popularity and
  capabilities. Leading this evolution are proprietary LLMs like GPT-4 and
  GPT-o1, which have captured widespread attention in the AI community due to
  their remarkable performance and versatility. Simultaneously, open-source LLMs,
  such as LLaMA and Mistral, have made great contributions to the ever-increasing
  popularity of LLMs due to the ease to customize and deploy the models across
  diverse applications. Although open-source LLMs present unprecedented
  opportunities for innovation and research, the commercialization of LLMs has
  raised concerns about transparency, reproducibility, and safety. Many
  open-source LLMs fail to meet fundamental transparency requirements by
  withholding essential components like training code and data, and some use
  restrictive licenses whilst claiming to be "open-source," which may hinder
  further innovations on LLMs. To mitigate this issue, we introduce Moxin 7B, a
  fully open-source LLM developed in accordance with the Model Openness Framework
  (MOF), a ranked classification system that evaluates AI models based on model
  completeness and openness, adhering to principles of open science, open source,
  open data, and open access. Our model achieves the highest MOF classification
  level of "open science" through the comprehensive release of pre-training code
  and configurations, training and fine-tuning datasets, and intermediate and
  final checkpoints. Experiments show that our model achieves superior
  performance in zero-shot evaluation compared with popular 7B models and
  performs competitively in few-shot evaluation.
paper_ai_summary: Moxin 7B is a fully open-source LLM that achieves high performance in zero-shot and few-shot evaluations while adhering to the Model Openness Framework principles.
paper_ai_keywords: Large Language Models LLMs GPT-4 GPT-o1 LLaMA Mistral Model Openness Framework MOF open science open source open data open access zero-shot evaluation few-shot evaluation