title: TeleChat Technical Report
summary: In this technical report, we present TeleChat, a collection of large language
  models (LLMs) with parameters of 3 billion, 7 billion and 12 billion. It
  includes pretrained language models as well as fine-tuned chat models that is
  aligned with human preferences. TeleChat is initially pretrained on an
  extensive corpus containing a diverse collection of texts from both English and
  Chinese languages, including trillions of tokens. Subsequently, the model
  undergoes fine-tuning to align with human preferences, following a detailed
  methodology that we describe. We evaluate the performance of TeleChat on
  various tasks, including language understanding, mathematics, reasoning, code
  generation, and knowledge-based question answering. Our findings indicate that
  TeleChat achieves comparable performance to other open-source models of similar
  size across a wide range of public benchmarks. To support future research and
  applications utilizing LLMs, we release the fine-tuned model checkpoints of
  TeleChat's 7B and 12B variant, along with code and a portion of our pretraining
  data, to the public community.
paper_authors._id: 659cc74abc65f1e59d02aa56
paper_authors.name: Zihan Wang
paper_authors._id: 659cc74abc65f1e59d02aa57
paper_authors.name: Xinzhang Liu
paper_authors._id: 659cc74abc65f1e59d02aa58
paper_authors.name: Shixuan Liu
paper_authors._id: 659cc74abc65f1e59d02aa59
paper_authors.name: Yitong Yao
paper_authors._id: 659cc74abc65f1e59d02aa5a
paper_authors.name: Yuyao Huang
paper_authors._id: 659cc74abc65f1e59d02aa5b
paper_authors.name: Zhongjiang He
paper_authors._id: 659cc74abc65f1e59d02aa5c
paper_authors.name: Xuelong Li
paper_authors._id: 659cc74abc65f1e59d02aa5d
paper_authors.name: Yongxiang Li
paper_authors._id: 659cc74abc65f1e59d02aa5e
paper_authors.name: Zhonghao Che
paper_authors._id: 659cc74abc65f1e59d02aa5f
paper_authors.name: Zhaoxi Zhang
paper_authors._id: 659cc74abc65f1e59d02aa60
paper_authors.name: Yan Wang
paper_authors._id: 659cc74abc65f1e59d02aa61
paper_authors.name: Xin Wang
paper_authors._id: 659cc74abc65f1e59d02aa62
paper_authors.name: Luwen Pu
paper_authors._id: 659cc74abc65f1e59d02aa63
paper_authors.name: Huihan Xu
paper_authors._id: 659cc74abc65f1e59d02aa64
paper_authors.name: Ruiyu Fang
paper_authors._id: 659cc74abc65f1e59d02aa65
paper_authors.name: Yu Zhao
paper_authors._id: 659cc74abc65f1e59d02aa66
paper_authors.name: Jie Zhang
paper_authors._id: 659cc74abc65f1e59d02aa67
paper_authors.name: Xiaomeng Huang
paper_authors._id: 659cc74abc65f1e59d02aa68
paper_authors.name: Zhilong Lu
paper_authors._id: 659cc74abc65f1e59d02aa69
paper_authors.name: Jiaxin Peng
paper_authors._id: 659cc74abc65f1e59d02aa6a
paper_authors.name: Wenjun Zheng
paper_authors._id: 659cc74abc65f1e59d02aa6b
paper_authors.name: Shiquan Wang
paper_authors._id: 659cc74abc65f1e59d02aa6c
paper_authors.name: Bingkai Yang
paper_authors._id: 659cc74abc65f1e59d02aa6d
paper_authors.name: Xuewei he
paper_authors._id: 659cc74abc65f1e59d02aa6e
paper_authors.name: Zhuoru Jiang
paper_authors._id: 659cc74abc65f1e59d02aa6f
paper_authors.name: Qiyi Xie
paper_authors._id: 659cc74abc65f1e59d02aa70
paper_authors.name: Yanhan Zhang
paper_authors._id: 659cc74abc65f1e59d02aa71
paper_authors.name: Zhongqiu Li
paper_authors._id: 659cc74abc65f1e59d02aa72
paper_authors.name: Lingling Shi
paper_authors._id: 659cc74abc65f1e59d02aa73
paper_authors.name: Weiwei Fu
paper_authors._id: 659cc74abc65f1e59d02aa74
paper_authors.name: Yin Zhang
paper_authors._id: 659cc74abc65f1e59d02aa75
paper_authors.name: Zilu Huang
paper_authors._id: 659cc74abc65f1e59d02aa76
paper_authors.name: Sishi Xiong
paper_authors._id: 659cc74abc65f1e59d02aa77
paper_authors.name: Yuxiang Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-05T10:20:45.386Z
paper_authors._id: 645b4a2978730bcc103dfe4d
paper_authors.avatarUrl: /avatars/de544de899897fd0a83506ff287123bc.svg
paper_authors.fullname: Yuxiang Zhang
paper_authors.type: user
paper_authors.user: TokerZ
paper_authors._id: 659cc74abc65f1e59d02aa78
paper_authors.name: Chao Wang
paper_authors._id: 659cc74abc65f1e59d02aa79
paper_authors.name: Shuangyong Song
paper_title: TeleChat Technical Report
paper_summary: In this technical report, we present TeleChat, a collection of large language
  models (LLMs) with parameters of 3 billion, 7 billion and 12 billion. It
  includes pretrained language models as well as fine-tuned chat models that is
  aligned with human preferences. TeleChat is initially pretrained on an
  extensive corpus containing a diverse collection of texts from both English and
  Chinese languages, including trillions of tokens. Subsequently, the model
  undergoes fine-tuning to align with human preferences, following a detailed
  methodology that we describe. We evaluate the performance of TeleChat on
  various tasks, including language understanding, mathematics, reasoning, code
  generation, and knowledge-based question answering. Our findings indicate that
  TeleChat achieves comparable performance to other open-source models of similar
  size across a wide range of public benchmarks. To support future research and
  applications utilizing LLMs, we release the fine-tuned model checkpoints of
  TeleChat's 7B and 12B variant, along with code and a portion of our pretraining
  data, to the public community.
paper_ai_summary: TeleChat, a series of large language models with varying parameter sizes, is pretrained on multilingual data and fine-tuned for human preference alignment, achieving comparable performance across multiple benchmarks.
paper_ai_keywords: large language models LLMs parameters pretrained language models fine-tuned chat models multilingual data human preferences language understanding mathematics reasoning code generation knowledge-based question answering model checkpoints pretraining data