Tele-FLM Technical Report
Large language models (LLMs) have showcased profound capabilities in language
understanding and generation, facilitating a wide array of applications.
However, there is a notable paucity of detailed, open-sourced methodologies on
efficiently scaling LLMs beyond 50 billion parameters with minimum
trial-and-error cost and computational resources. In this report, we introduce
Tele-FLM (aka FLM-2), a 52B open-sourced multilingual large language model that
features a stable, efficient pre-training paradigm and enhanced factual
judgment capabilities. Tele-FLM demonstrates superior multilingual language
modeling abilities, measured by BPB on textual corpus. Besides, in both English
and Chinese foundation model evaluation, it is comparable to strong
open-sourced models that involve larger pre-training FLOPs, such as Llama2-70B
and DeepSeek-67B. In addition to the model weights, we share the core designs,
engineering practices, and training details, which we expect to benefit both
the academic and industrial communities.
662b17cee5d910dd729834f9
Xiang Li
claimed_verified
2024-04-26T10:05:34.081Z
6486ea539264d020ddb2a06e
/avatars/3d5f2e8e13c87ad89197f171395d9556.svg
Xiang Li
user
sleepylx
662b17cee5d910dd729834fa
Yiqun Yao
admin_assigned
2024-04-26T09:45:35.744Z
64f9b61f0c7dbab99d9b0c16
/avatars/fe5e09578cdebc97c970e6795c836a49.svg
Yiqun Yao
user
yqyao
662b17cee5d910dd729834fb
Xin Jiang
admin_assigned
2024-04-26T09:45:55.068Z
647415007afa69c3c7a98f1f
https://cdn-avatars.huggingface.co/v1/production/uploads/647415007afa69c3c7a98f1f/pEl0PozmzNK8_PwUMiikd.jpeg
Xin Jiang
user
horiz94
662b17cee5d910dd729834fc
Xuezhi Fang
claimed_verified
2024-04-28T19:30:11.025Z
63c4f5588d95a5c770700ff8
/avatars/4f64029f74cba179d263c0e4f1887427.svg
Jason Fang
user
jasonfang3900
662b17cee5d910dd729834fd
Chao Wang
662b17cee5d910dd729834fe
Xinzhang Liu
662b17cee5d910dd729834ff
Zihan Wang
claimed_verified
2025-07-25T08:40:45.680Z
640ad946b871d7117d5eead7
/avatars/4cd90f6926a24c58485ef364eae50ef1.svg
Zihan Wang
user
ZihanWang99
662b17cee5d910dd72983500
Yu Zhao
662b17cee5d910dd72983501
Xin Wang
662b17cee5d910dd72983502
Yuyao Huang
662b17cee5d910dd72983503
Shuangyong Song
662b17cee5d910dd72983504
Yongxiang Li
admin_assigned
2024-04-26T10:03:54.318Z
660b9ef3f4ab651901147194
/avatars/12720ddf21084f9ec48313d156918d63.svg
yongxiang
user
liyongxiang
662b17cee5d910dd72983505
Zheng Zhang
662b17cee5d910dd72983506
Bo Zhao
admin_assigned
2024-04-26T10:06:14.593Z
64acafac1aee69ece03af05a
/avatars/6380777852546dbd432fed5544d5db0b.svg
Bo Zhao
user
Z-LIRA
662b17cee5d910dd72983507
Aixin Sun
662b17cee5d910dd72983508
Yequan Wang
admin_assigned
2024-04-26T10:06:38.480Z
64f6db81867bbeac42b22b5d
/avatars/cb945f3fce8f3b77f0316c83b1496a29.svg
Wang Yequan
user
thuwyq
662b17cee5d910dd72983509
Zhongjiang He
662b17cee5d910dd7298350a
Zhongyuan Wang
662b17cee5d910dd7298350b
Xuelong Li
662b17cee5d910dd7298350c
Tiejun Huang
Tele-FLM Technical Report
Large language models (LLMs) have showcased profound capabilities in language
understanding and generation, facilitating a wide array of applications.
However, there is a notable paucity of detailed, open-sourced methodologies on
efficiently scaling LLMs beyond 50 billion parameters with minimum
trial-and-error cost and computational resources. In this report, we introduce
Tele-FLM (aka FLM-2), a 52B open-sourced multilingual large language model that
features a stable, efficient pre-training paradigm and enhanced factual
judgment capabilities. Tele-FLM demonstrates superior multilingual language
modeling abilities, measured by BPB on textual corpus. Besides, in both English
and Chinese foundation model evaluation, it is comparable to strong
open-sourced models that involve larger pre-training FLOPs, such as Llama2-70B
and DeepSeek-67B. In addition to the model weights, we share the core designs,
engineering practices, and training details, which we expect to benefit both
the academic and industrial communities.
An open-sourced 52B multilingual large language model, Tele-FLM, features efficient pre-training and superior language capabilities, and is comparable to larger models in terms of performance metrics.
LLMs large language models pre-training paradigm factual judgment capabilities BPB multilingual language modeling English foundation model Chinese foundation model FLOPs Llama2-70B DeepSeek-67B