TeacherLM: Teaching to Fish Rather Than Giving the Fish, Language
  Modeling Likewise
Large Language Models (LLMs) exhibit impressive reasoning and data
augmentation capabilities in various NLP tasks. However, what about small
models? In this work, we propose TeacherLM-7.1B, capable of annotating relevant
fundamentals, chain of thought, and common mistakes for most NLP samples, which
makes annotation more than just an answer, thus allowing other models to learn
"why" instead of just "what". The TeacherLM-7.1B model achieved a zero-shot
score of 52.3 on MMLU, surpassing most models with over 100B parameters. Even
more remarkable is its data augmentation ability. Based on TeacherLM-7.1B, we
augmented 58 NLP datasets and taught various student models with different
parameters from OPT and BLOOM series in a multi-task setting. The experimental
results indicate that the data augmentation provided by TeacherLM has brought
significant benefits. We will release the TeacherLM series of models and
augmented datasets as open-source.
65408bfa99e887c0405a6956
Nan He
65408bfa99e887c0405a6957
Hanyu Lai
admin_assigned
2023-10-31T09:42:47.526Z
62ea2d48b493272c269e8f34
/avatars/89b215dafa503b51ab212a9b63c82aca.svg
Hanyu Lai
user
hanyullai
65408bfa99e887c0405a6958
Chenyang Zhao
admin_assigned
2023-10-31T09:42:58.413Z
64d457059ba46988ac54d40e
/avatars/4b12a2cda302c38f931dd7555c233daa.svg
Chenyang Zhao
user
cyzhao
65408bfa99e887c0405a6959
Zirui Cheng
65408bfa99e887c0405a695a
Junting Pan
admin_assigned
2023-10-31T09:43:37.893Z
639c2d36e62c431aa8d834a7
/avatars/a23d57dcb190711494a2a7e75e4cb9a8.svg
Junting Pan
user
juntingpan
65408bfa99e887c0405a695b
Ruoyu Qin
65408bfa99e887c0405a695c
Ruofan Lu
admin_assigned
2023-10-31T09:43:52.843Z
63203f0a7f395c502abcfcde
/avatars/e6a4df28ce3fc7c6a5cfbbd60bf5563a.svg
Ruofan Lu
user
lurf21
65408bfa99e887c0405a695d
Rui Lu
admin_assigned
2023-11-03T09:03:11.381Z
63ca0433b047b9b12a1fcd9c
/avatars/435507115bfc502c494989677cfca5fb.svg
Rui Lu
user
learning-rate
65408bfa99e887c0405a695e
Yunchen Zhang
admin_assigned
2023-10-31T09:44:18.803Z
6204bd1e12ccba127e39aa94
https://cdn-avatars.huggingface.co/v1/production/uploads/1663764189519-6204bd1e12ccba127e39aa94.jpeg
Yunchen Zhang
user
yunchen
65408bfa99e887c0405a695f
Gangming Zhao
65408bfa99e887c0405a6960
Zhaohui Hou
65408bfa99e887c0405a6961
Zhiyuan Huang
admin_assigned
2023-10-31T09:44:59.620Z
64f5a2e46a89a9fc0dc6f470
https://cdn-avatars.huggingface.co/v1/production/uploads/64f5a2e46a89a9fc0dc6f470/L0thrI2El7RhXCf6W0gVz.jpeg
huangzhiyuan
user
huangzhiyuan
65408bfa99e887c0405a6962
Shaoqing Lu
65408bfa99e887c0405a6963
Ding Liang
admin_assigned
2023-10-31T09:45:13.971Z
6476b01cbb7fdd7f425dcefb
/avatars/65efe7067f47a68c204a1ab5a772b939.svg
dingliang
user
dingliang01
65408bfa99e887c0405a6964
Mingjie Zhan
TeacherLM: Teaching to Fish Rather Than Giving the Fish, Language
  Modeling Likewise
Large Language Models (LLMs) exhibit impressive reasoning and data
augmentation capabilities in various NLP tasks. However, what about small
models? In this work, we propose TeacherLM-7.1B, capable of annotating relevant
fundamentals, chain of thought, and common mistakes for most NLP samples, which
makes annotation more than just an answer, thus allowing other models to learn
"why" instead of just "what". The TeacherLM-7.1B model achieved a zero-shot
score of 52.3 on MMLU, surpassing most models with over 100B parameters. Even
more remarkable is its data augmentation ability. Based on TeacherLM-7.1B, we
augmented 58 NLP datasets and taught various student models with different
parameters from OPT and BLOOM series in a multi-task setting. The experimental
results indicate that the data augmentation provided by TeacherLM has brought
significant benefits. We will release the TeacherLM series of models and
augmented datasets as open-source.
TeacherLM-7.1B significantly enhances NLP tasks through zero-shot performance and data augmentation, surpassing larger models and improving various student models.
Large Language Models NLP tasks TeacherLM-7.1B chain of thought common mistakes MMLU data augmentation OPT series BLOOM series multi-task setting zero-shot score